Real-Time Depth Estimation with an Optimized Encoder-Decoder Architecture on Embedded Devices

As a strategy to obtain dense depth maps from a single image, sparse depths, or the fusion of both (RGBd), depth estimation receives much attention. Usually relying on high-performance workstations, existing depth prediction methods, are not particularly suitable for embedded devices. Moreover, the accuracy of depth estimation needs to be improved. Towards real-time and reliable depth prediction on portable embedded platforms, we propose an encoder-decoder architecture (EDA). Specifically, these are the three contributions of this work. (1) We present a light-weight encoder, which takes three modalities of data as inputs, namely, RGB images, sparse depths, or RGBd. The encoder is efficient to extract features on portable devices. (2) A novel decoder is designed to obtain accurate and high-resolution depth maps. (3) We use the existing TVM to deploy, compile, and optimize EDA on embedded platforms, for further improvements in the computational speed and power of depth estimation. In real scenarios, the optimized EDA achieves remarkable results with low latency. Through extensive experimental evaluation on two public datasets, we show that the optimized EDA outperforms the state of the arts in accuracy and computational resources. To the best of our knowledge, this is the first paper that provides an efficient encoder, learnable decoder, and end-to-end optimized encoder-decoder architecture for precise depth prediction.