Domain adaptation of deep neural networks for tree part segmentation using synthetic forest trees

Supervised deep learning algorithms have recently achieved state-of-the-art performance in the classification, segmentation and analysis of 3D LiDAR point cloud data in a wide-range of applications and environments. One of the main downsides of deep learning-based approaches is the need for extensive training datasets, i.e. LiDAR point clouds that have been annotated for target tasks by human experts. One strategy for addressing this issue is the use of simulated/synthetic data (with automatically generated annotations) for training models which can then be deployed on real target data/environments. This paper explores using synthetic data of realistic forest trees to train deep learning models for tree part segmentation from real forest LiDAR data. We develop a new pipeline for generating high-fidelity simulated LiDAR scans of synthetic forest trees and combine this with an unsupervised domain adaptation strategy to adapt models trained on synthetic data to LiDAR data captured in real forest environments. Models were trained for semantic segmentation of tree parts using a PointNet++ architecture and evaluated across a range of medium to high-resolution laser scanning datasets collected across both ground-based and aerial platforms in multiple forest environments. Results of our work indicated that models trained on our synthetic data pipeline were competitive with models trained on real data, in particular when real data came from non-target sites, and our unsupervised domain adaptation method further improved performance. Our approach has implications for reducing the burden required in manual human expert annotation of large LiDAR datasets required to achieve high-performance from deep learning methods for forest analysis. The use of synthetically-trained models shown here provides a potential way to reduce the barriers to the use of deep learning in large-scale forest analysis, with implications to applications ranging from forest inventories to scaling-up in-situ forest phenotyping.