Extreme View Synthesis: Novel Techniques and Challenges

Explore the concept of extreme view synthesis in this 28-minute lecture from the University of Central Florida. Delve into novel view synthesis techniques, including stereo magnification and DeepMVS. Learn about multiplane images, patch matching, and inter-volume feature aggregation. Understand the challenges addressed by extreme view synthesis and examine the proposed solution and architecture. Discover methods for estimating depth probability volumes, handling depth discontinuities, and synthesizing novel views. Analyze artifacts in synthesized images and learn about patch warping and refinement techniques. Evaluate the approach through quantitative results and comparisons with other methods. Conclude by examining the limitations and potential applications of extreme view synthesis in computer vision and graphics.

Outline
What is Novel View Synthesis?
Stereo Magnification: Learning view synthesis using multiplane images, Zhou et al.
Stereo Magnification: Multiplane Image
DeepMVS: Learning multi-view stereopsis, Huang et al.
DeepMVS: Patch Matching
DeepMVS: Inter-Volume Feature Aggregation
DeepMVS: Disparity Maps
What is trying to be solved?
Proposed Solution
Proposed Architecture
Estimating the Depth Probability Volume
Depth Probability Distributions and Discontinuities
Estimating Depth Probability of the Novel Vie
Synthesis of a Novel View
Artifacts Present After Synthesis
Patch Warping
Refinement Network Training
Refinement Example
Evaluation
Quantitative Results
Comparison to Stereo Magnification
Comparison to Soft3D
Limitations
Conclusion