Instant Mixed Reality Lighting from Casual Scanning

Thomas Richter-Trummer, Denis Kalkofen, Jinwoo Park, Dieter Schmalstieg

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

35 Citations (Scopus)

Abstract

We present a method for recovering both incident lighting and surface materials from casually scanned geometry. By casual, we mean a rapid and potentially noisy scanning procedure of unmodified and uninstrumented scenes with a commodity RGB-D sensor. In other words, unlike reconstruction procedures which require careful preparations in a laboratory environment, our method works with input that can be obtained by consumer users. To ensure a robust procedure, we segment the reconstructed geometry into surfaces with homogeneous material properties and compute the radiance transfer on these segments. With this input, we solve the inverse rendering problem of factorization into lighting and material properties using an iterative optimization in spherical harmonics form. This allows us to account for self-shadowing and recover specular properties. The resulting data can be used to generate a wide range of mixed reality applications, including the rendering of synthetic objects with matching lighting into a given scene, but also re-rendering the scene (or a part of it) with new lighting. We show the robustness of our approach with real and synthetic examples under a variety of lighting conditions and compare them with ground truth data.

Original languageEnglish
Title of host publicationProceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016
EditorsWolfgang Broll, Hideo Saito, J. Edward Swan
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages27-36
Number of pages10
ISBN (Electronic)9781509036417
DOIs
Publication statusPublished - 12 Dec 2016
Externally publishedYes
Event15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016 - Merida, Yucatan, Mexico
Duration: 19 Sept 201623 Sept 2016

Publication series

NameProceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016

Conference

Conference15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016
Country/TerritoryMexico
CityMerida, Yucatan
Period19/09/1623/09/16

Keywords

  • [Information Interfaces and Presentation
  • Artificial
  • augmented
  • virtual realities
  • Image Processing and Computer Vision
  • Photometric registration
  • 3D Reconstruction

Fingerprint

Dive into the research topics of 'Instant Mixed Reality Lighting from Casual Scanning'. Together they form a unique fingerprint.

Cite this