

demo 
arXiv 
code 
video 

(A): a photo (credit David Clapp) with an offcentered principal point. (B), (C): assuming traditional pinhole model with principal point at the center, there is no way to correctly represent both up directions (wrong in B) and horizon (wrong in C). (D): Our proposed Perspective Fields model correctly models the Upvectors (Green arrows) aligned with gravity, and Latitude values (contour line: π/2 π/2) with 0° on the horizon. We can further recover camera parameters Roll 0.5°, Pitch 1.7°, FoV 64.6° and principal point at × from the prediction. 
Geometric camera calibration is essential for applications that require understanding image perspective. We introduce Perspective Fields, a representation modeling local perspective properties using perpixel information about the camera view, parameterized by an up vector and latitude value. This approach offers several benefits, such as minimal assumptions about the camera model, invariance or equivariance to common image editing operations, and improved interpretability aligned with human perception. We train a neural network to predict Perspective Fields, which can be easily converted to calibration parameters. Our method outperforms camera calibrationbased approaches in various scenarios and is demonstrated in image compositing applications.
Check out how Perspective Fields change w.r.t. pinhole camera parameters.
Roll
Pitch
FoV

For each pixel location, the Perspective Field consists of a unit Upvector and Latitude. The Upvector is the projection of the up direction, shown in Green arrows. In perspective projection, it points to the vertical vanishing point. The Latitude of each pixel is defined as the angle between the incoming ray and the horizontal plane. We show it using contour line: π/2 π/2. Note 0° is at the horizon.
Experience our Gradio demo for Perspective Fields and camera parameter prediction. Choose from uncurated examples or upload your own image. Keep in mind that it runs on a CPU, resulting in slower performance. For faster inference, please clone our code and run it on a GPU.
We can train a neural network to predict Perspective Fields from images in the wild. Below are some examples.
Input 

Perspective Fields 































































Moreover, it generalizes to nonperspective projections, such as the multiperspective scene from Inception or artworks with various camera models.

Linyi Jin, Jianming Zhang, Yannick HoldGeoffroy, Oliver Wang, Kevin Matzen, Matthew Sticha, David F. Fouhey Perspective Fields for Single Image Camera Calibration. CVPR 2023. (Paper) 
AcknowledgementsThis work was partially funded by the DARPA Machine Common Sense Program. This webpage template originally made from some colorful folks. 