PanoWorld Real-World Panoramic Generation

Haoyuan Li¹, Dizhe Zhang¹, Yuemei Zhou¹, Xiangkai Zhang^1,2, Haoran Feng^1,3 Xiaofan Lin¹, Wenjie Jiang¹, Bo Du⁴, Ming-Hsuan Yang⁵, Lu Qi^1,4

¹Insta360 Research ²Institute of Automation Chinese Academy of Sciences
³Tsinghua University ⁴Wuhan University ⁵UC Merced

Paper Code Models World360 · Coming Soon

Scroll to explore

Overview

Abstract

In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation. Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA). Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360. Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin. Our models, training code, and dataset will be publicly available.

Qualitative Comparison

Click video to open VR mode

480P Results

The model generates an 81-frame panoramic video at 480 × 960 resolution.

Click video to open VR mode

720P Results

Click video to open VR mode

Model Design & Training Strategy

PanoWorld Network Architecture. Built on the Wan2.2 backbone, PanoWorld employs a triple-stream DiT that fuses visual self-attention, DPRC-based action modeling, and a GMA module for memory-anchored synthesis via a shared geometric manifold.

Overview of the progressive three-stage pipeline. Stage 1 focuses on geometric adaptation for panoramic video generation; Stage 2 targets view-dependent motion learning; and Stage 3 enables memory-anchored coherence to enforce long-term temporal consistency.

World360 Benchmark

^‡ Existing large-scale real datasets primarily consist of planar street-level motion, where altitude information is ambiguous or fixed. In contrast, World360 provides diverse aerial trajectories across multiple altitudes.

Panoramic clips from World360, spanning real-world UAV captures and AirSim360 simulated environments.

The Data Curation Pipeline. The pipeline converts raw panoramic clips into high-quality sequences by: (1) Rotation Decoupling to isolate pure translation; (2) Uniform Spatial Resampling to standardize motion scales; and (3) Illumination Filtering to ensure exposure consistency.

Memory Modeling

Qualitative ablation results of the GMA module. Comparison of synthesized video frames under control: (a) w/o GMA baseline exhibits significant artifacts; (b) Random Memory variant results in broken geometry; (c) Full setting with GMA maintains consistent generation.

Ablation study on the Memory Module. We compare trajectory control stability (PSNR) across different temporal windows. Bold indicates the best performance.

Quantitative Results

Quantitative results on panoramic video generation. Best results are in red.

Motion control evaluation. Best results are in red, while `--` denotes metrics not supported by the baseline's original implementation.

Qualitative Comparison Evaluation of Trajectory Fidelity via ViPE. We utilize ViPE to reconstruct camera trajectories from the synthesized videos, comparing them against the GT.

Extensions

Causal-Forcing Real-time Generation

To achieve real-time panoramic video generation, PanoWorld employs a self-forcing distillation technique that compresses the full model into a lightweight autoregressive previewer. By using the model's own predictions as conditioning for subsequent frames and matching the teacher model's distribution, this approach drastically accelerates generation speed—producing an 81-frame sequence in mere seconds—while successfully preserving the overall scene structure.

The top row shows the input first frame, followed by generated frames along the specified camera trajectories.

Citation

@article{panoworld2026,
  title   = {PanoWorld: Real-World Panoramic Generation},
  
}