URoPE: Universal Relative Position Embedding across Geometric Spaces

Relative position embedding has become a standard mech-anism for encoding positional information in Transformers. However,existing formulations are typically limited to a fixed geometric space,namely 1D sequences or regular 2D/3D grids, which restricts their ap-plicability to many computer vision tasks that require geometric rea-soning across camera views or between 2D and 3D spaces. To addressthis limitation, we propose URoPE, a universal extension of RotaryPosition Embedding (RoPE) to cross-view or cross-dimensional geomet-ric spaces. For each key/value image patch, URoPE samples 3D pointsalong the corresponding camera ray at predefined depth anchors andprojects them into the query image plane. Standard 2D RoPE can thenbe applied using the projected pixel coordinates. URoPE is a parameter-free and intrinsics-aware relative position embedding that is invariant tothe choice of global coordinate systems, while remaining fully compatiblewith existing RoPE-optimized attention kernels. We evaluate URoPE asa plug-in positional encoding for transformer architectures across a di-verse set of tasks, including novel view synthesis, 3D object detection,object tracking, and depth estimation, covering 2D–2D, 2D–3D, and tem-poral scenarios. Experiments show that URoPE consistently improvesthe performance of transformer-based models across all tasks, demon-strating its effectiveness and generality for geometric reasoning. Our codeis available on our project website: https://urope-pe.github.io/.