Perspective Fields for Single Image Camera Calibration

1University of Michigan

2Adobe Research

CVPR 2023 (Highlight)

Demo Icon
Code Icon
Code Icon
Video Icon

(A): a photo (credit David Clapp) with an off-centered principal point. (B), (C): assuming traditional pinhole model with principal point at the center, there is no way to correctly represent both up directions (wrong in B) and horizon (wrong in C). (D): Our proposed Perspective Fields model correctly models the Up-vectors (Green arrows) aligned with gravity, and Latitude values (contour line: -π/2 π/2) with 0° on the horizon. We can further recover camera parameters Roll -0.5°, Pitch 1.7°, FoV 64.6° and principal point at × from the prediction.

Geometric camera calibration is essential for applications that require understanding image perspective. We introduce Perspective Fields, a representation modeling local perspective properties using per-pixel information about the camera view, parameterized by an up vector and latitude value. This approach offers several benefits, such as minimal assumptions about the camera model, invariance or equivariance to common image editing operations, and improved interpretability aligned with human perception. We train a neural network to predict Perspective Fields, which can be easily converted to calibration parameters. Our method outperforms camera calibration-based approaches in various scenarios and is demonstrated in image compositing applications.


Perspective Fields on Pinhole Camera

Check out how Perspective Fields change w.r.t. pinhole camera parameters.

Roll      0
Pitch   0
FoV    70

For each pixel location, the Perspective Field consists of a unit Up-vector and Latitude. The Up-vector is the projection of the up direction, shown in Green arrows. In perspective projection, it points to the vertical vanishing point. The Latitude of each pixel is defined as the angle between the incoming ray and the horizontal plane. We show it using contour line: -π/2 π/2. Note 0° is at the horizon.

Interactive Demo

Experience our Gradio demo for Perspective Fields and camera parameter prediction. Choose from uncurated examples or upload your own image. Keep in mind that it runs on a CPU, resulting in slower performance. For faster inference, please clone our code and run it on a GPU.

Qualitative Results

We can train a neural network to predict Perspective Fields from images in the wild. Below are some examples.


Perspective Fields

Moreover, it generalizes to non-perspective projections, such as the multi-perspective scene from Inception or artworks with various camera models.

Paper and Supplementary Material

Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Matzen, Matthew Sticha, David F. Fouhey
Perspective Fields for Single Image Camera Calibration.
CVPR 2023.



This work was partially funded by the DARPA Machine Common Sense Program. This webpage template originally made from some colorful folks.