Object Recognition Using a Sparse 3D Camera Point Cloud

Abstract

The demand for higher precision and speed of computer vision models is increasing in autonomous driving, robotics, smart city and numerous other applications. In that context, machine learning is gaining increasing attention as it enables a more comprehensive understanding of the environment. More reliable and accurate imaging sensors are needed to maximise the performance of machine learning models. One example of a new sensor is LightCode Photonics’ 3D camera. The thesis presents a study to evaluate the performance of machine learning-based object recognition in an urban environment using a relatively low spatial resolution 3D camera. As the angular resolution of the camera is smaller than in commonly used 3D imaging sensors, using the camera output with already published object recognition models makes the thesis unique and valuable for the company, providing feedback for LightCode Photonics’ current camera specifications for machine learning tasks. Furthermore, the knowledge and materials could be used to develop the company’s object recognition pipeline. During the thesis, a new dataset is generated in CARLA Simulator and annotated, representing the 3D camera in a smart city application. Changes to CARLA Simulator source code were implemented to represent the actual camera closely. The thesis is finished with experiments where PointNet semantic segmentation and PointPillars object detection models are applied to the generated dataset. The generated dataset contained 4599 frames, of which 2816 were decided to use in this thesis. PointNet model applied to the dataset could predict the semantically segmented scene with similar accuracy as in the original paper. A mean accuracy of 44.15% was achieved with PointNet model. On the other hand, PointPillars model was unable to perform on the new dataset.