A real-time interactive system of surface reconstruction and dynamic projection mapping with RGB-depth sensor and projector

Yundong Guo, Shu Chuan Chu, Zhenyu Liu, Chan Qiu, Hao Luo, Jianrong Tan

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)
66 Downloads (Pure)

Abstract

Reconstruction and projection mapping enable us to bring virtual worlds into real spaces, which can give spectators an immersive augmented reality experience. Based on an interactive system with RGB-depth sensor and projector, we present a combined hardware and software solution for surface reconstruction and dynamic projection mapping in real time. In this article, a novel and adaptable calibration scheme is proposed, which is used to estimate approximate models to correct and transform raw depth data. Besides, our system allows for smooth real-time performance using an optimization framework, including denoising and stabilizing. In the entire pipeline, markers are only used in the calibration procedure, and any priors are not needed. Our approach enables us to interact with the target surface in real time, while maintaining correct illumination. It is easy and fast to develop different applications for our system, and some interesting cases are demonstrated at last.

Original languageEnglish
JournalInternational Journal of Distributed Sensor Networks
Volume14
Issue number7
DOIs
Publication statusPublished - 27 Jul 2018

Bibliographical note

Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License(http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work withoutfurther permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage)

Keywords

  • Augmented reality
  • interactive system
  • projection mapping
  • RGB-depth sensor
  • surface reconstruction

Fingerprint

Dive into the research topics of 'A real-time interactive system of surface reconstruction and dynamic projection mapping with RGB-depth sensor and projector'. Together they form a unique fingerprint.

Cite this