Neural cameras: Learning camera characteristics for coherent mixed reality rendering

David Mandl, Peter Mohr, Tobias Langlotz, Christoph Ebner, Shohei Mori, Stefanie Zollmann, Peter M. Roth, Denis Kalkofen

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

1 Citation (Scopus)

Abstract

Coherent rendering is important for generating plausible Mixed Reality presentations of virtual objects within a user’s real-world environment. Besides photo-realistic rendering and correct lighting, visual coherence requires simulating the imaging system that is used to capture the real environment. While existing approaches either focus on a specific camera or a specific component of the imaging system, we introduce Neural Cameras, the first approach that jointly simulates all major components of an arbitrary modern camera using neural networks. Our system allows for adding new cameras to the framework by learning the visual properties from a database of images that has been captured using the physical camera. We present qualitative and quantitative results and discuss future direction for research that emerge from using Neural Cameras.

Original languageEnglish
Title of host publication2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
EditorsMaud Marchal, Jonathan Ventura, Anne-Helene Olivier, Lili Wang, Rafael Radkowski
Place of PublicationNew Jersey, U.S.A.
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages508-516
Number of pages9
ISBN (Electronic)9781665401586, 9781728197777
DOIs
Publication statusPublished - 2021
Externally publishedYes
Event20th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021 - Virtual, Online, Italy
Duration: 4 Oct 20218 Oct 2021

Publication series

NameInternational Symposium on Mixed and Augmented Reality (ISMAR). Proceedings
PublisherI E E E Computer Society
ISSN (Print)1554-7868

Conference

Conference20th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021
Country/TerritoryItaly
CityVirtual, Online
Period4/10/218/10/21

Keywords

  • Camera Characteristics
  • virtual objects
  • photo-realistic rendering
  • Neural Cameras

Fingerprint

Dive into the research topics of 'Neural cameras: Learning camera characteristics for coherent mixed reality rendering'. Together they form a unique fingerprint.

Cite this