Global Pose Control for Generative View Synthesis
in Normalized Object Coordinate Space

Zhibing Li1,† Amogh Gupta2 Behnoosh Parsa2 Dan Casas2
1 The Chinese University of Hong Kong, China 2 Amazon, USA
† Work done while Zhibing Li was an intern at Amazon.

Abstract

Novel View Synthesis (NVS) enables the generation of unseen views of a scene from a single or multiple images, allowing users to freely explore an object from any viewpoint. Despite the recent impressive qualitative improvements of generative models for this task, existing methods struggle to provide global and intuitive control of target viewpoints because they either use input-relative camera poses or are limited to generating sparse global views. This lack of global pose control severely limits the number of downstream tasks potentially enabled by NVS.

To address this limitation, we propose a novel approach for precise camera control in a customizable Normalized Object Coordinate Space (NOCS), requiring single or few unposed images. Our method operates solely on the absolute camera pose of the target view in NOCS, eliminating the need for a relative world frame or camera poses of the input images. Unlike previous methods that treat NVS as a standalone generation task, we formulate it as an image editing problem and build upon state-of-the-art editing models to leverage their superior generalization capability. Camera information is injected as dedicated camera tokens via an in-context multi-modal conditioning strategy. To alleviate the inherent ambiguity of NOCS, we incorporate text descriptions that explicitly define the object's canonical coordinate frame, which also enhances generalization to unseen object categories. Furthermore, we curate a high-quality dataset with consistently aligned orientations and corresponding NOCS text definitions.

Extensive experiments demonstrate that our method robustly generates novel views with accurate and consistent orientations from arbitrary unposed images across diverse categories, achieving state-of-the-art image quality and fidelity.

Given unposed input images and target camera poses, our model generates high-quality novel views with consistent global orientation.
Given unposed input images captured from arbitrary viewpoints (left) and target camera poses (top), our model generates high-quality novel views with consistent global orientation across diverse object families — the front of each object stays aligned with the red axis.

Method

Reframe NVS as Image Editing

Unlike prior methods that treat novel view synthesis as a standalone generation task, we formulate it as an image editing problem, where the target views are obtained by spatially transforming the input observations. This allows us to build upon a state-of-the-art image editing model and directly inherit its high image quality and strong generalization to in-the-wild objects.

Control in NOCS

We perform camera control directly in a customizable Normalized Object Coordinate Space (NOCS). Given one or more unposed input images, our model conditions on the absolute camera pose of each target view in NOCS, removing the need for a reference view or a relative world frame. A text prompt defines the object's canonical orientation, enabling intuitive global viewpoint control that generalizes to unseen categories.

Overview of our pipeline: target camera poses are encoded as Plücker ray map tokens and fed into a LoRA-adapted MMDiT alongside image tokens, with a text prompt defining the object's canonical orientation in NOCS.
Overview. Left: Target camera poses are encoded as Plücker ray map tokens and fed into a LoRA-adapted MMDiT alongside image tokens; a text prompt defines the object's canonical orientation in NOCS. Top-right: Target images and ray map tokens are packed along the frame axis, with ray map position embeddings interpolated to spatially align with the corresponding image tokens. Bottom-right: Regional attention ensures each camera token attends only to its corresponding target view, while all image tokens retain full mutual attention.

Gallery

Generated novel views with consistent global orientation across three benchmarks. Click each panel to expand. Use the controls to change the number of input views, and toggle inputs or ground truth. Click any image to view it full size.

Orientation Reference

Orientation Reference

Input Scene

Real-world input scene with multiple objects

Orientation Reference

Prediction Results

Existing 3D reconstruction methods such as FreeSplatter can generate 3D representations (e.g., Gaussian splats) from a sparse set of input views. Our method generates additional views whose absolute orientations are known in a normalized coordinate space. By augmenting the input with these views, the reconstruction is canonicalized with respect to the normalized frame.

BibTeX