Real-Time Orbital Image Analysis Using Decision Forests, with a Deployment Onboard the IPEX Spacecraft

Automatic cloud recognition promises significant improvements in Earth science remote sensing. At any time, more than half of Earth's surface is covered by clouds, obscuring images and atmospheric measurements. This is particularly problematic for CubeSats, a new generation of small, low-orbiting spacecraft with very limited communications bandwidth. Such spacecraft can use image analysis to autonomously select clear scenes for prioritized downlink. More agile spacecraft can also benefit from cloud screening by retargeting observations to cloud-free areas. This could significantly improve the science yield of instruments such as the Orbiting Carbon Observatory 3 mission. However, most existing cloud detection algorithms are not suitable for these applications, because they require calibrated and georectified spectral data, which is not typically available onboard. Here, we describe a statistical machine-learning method for real-time autonomous scene interpretation using a visible camera with no radiometric calibration. A random forest classifies cloud and clear pixels based on local patterns of image texture. We report on experimental evaluation of images from the International Space Station ISS and present results from a deployment onboard the IPEX spacecraft. This demonstrates actual execution in flight and provides some preliminary lessons learned about operational use. It is a rare example of a machine-learning system deployed to an autonomous spacecraft. To our knowledge, it is also the first instance of significant artificial intelligence deployed on board a CubeSat and the first ever deployment of visible image-based cloud screening onboard any operational spacecraft.