Automation of Recording in Smart Classrooms via Deep Learning and Bayesian Maximum a Posteriori Estimation of Instructor's Pose

Mohammad Sayad Haghighi, Alireza Sheikhjafari, Alireza Jolfaei, Faezeh Farivar, Sahar Ahmadzadeh

Research output: Contribution to journalArticlepeer-review

11 Citations (Scopus)

Abstract

Internet of Things is making objects smarter and more autonomous. At the other side, online education is gaining momentum and many universities are now offering online degrees. Content preparation for such programs usually involves recording the classes. In this article, we intend to introduce a deep learning-based camera management system as a substitute for the academic filming crew. The solution mainly consists of two cameras and a wearable gadget for the instructor. The fixed camera is used for the instructor's position and pose detection and the pan-tilt-zoom (PTZ) camera does the filming. In the proposed solution, image processing and deep learning techniques are merged together. Face recognition and skeleton detection algorithms are used to detect the position of instructor. But the main contribution lies in the application of deep learning for instructor's skeleton detection and postprocessing of the deep network output for correction of the pose detection results using a Bayesian Maximum A Posteriori (MAP) estimator. This estimator is defined on a Markov state machine. The pose detection result along with the position info is then used by the PTZ camera controller for filming purposes. The proposed solution is implemented by using OpenPose which is a convolutional neural network for detection of body parts. Feeding a neural network pose classifier with 12 features extracted from the output of the deep network yields an accuracy of 89%. However, as we show, the accuracy can be improved by the Markov model and MAP estimator to reach as high as 95.5%.

Original languageEnglish
Article number9147024
Pages (from-to)2813-2820
Number of pages8
JournalIEEE Transactions on Industrial Informatics
Volume17
Issue number4
DOIs
Publication statusPublished - Apr 2021
Externally publishedYes

Keywords

  • Bayesian estimation
  • classification
  • convolutional neural network (CNN)
  • deep learning
  • Internet of Things (IoT)
  • Markov model
  • OpenPose
  • smart classroom

Fingerprint

Dive into the research topics of 'Automation of Recording in Smart Classrooms via Deep Learning and Bayesian Maximum a Posteriori Estimation of Instructor's Pose'. Together they form a unique fingerprint.

Cite this