Analyzing multi–domain learning for enhanced rockfall mapping in known and unknown planetary domains

Abstract Rockfalls are small–scale mass wasting events that have been observed across the solar system. They provide valuable information about the endo- and exogenic activity of their host body, but are difficult to identify and map in satellite imagery, especially on global scales and in big data sets. Past work implemented convolutional neural networks to automate rockfall mapping on the Moon and Mars with the caveat of (1) achieving sub–optimal performance and (2) requiring substantial manual image labeling efforts. Mixing annotated image data from the Moon and Mars while keeping the total number of labels constant, we show that including a small number (10%) of rockfall labels from a foreign domain (e.g. Moon) during detector training can increase performance in the home domain (e.g. Mars) by up to 6% Average Precision (AP) in comparison to a purely home domain-trained detector. We additionally show that using a large number of foreign domain training examples (90%) in combination with a small number (10%) of home domain labels can be as powerful or more powerful as exclusively (100%) using home labels in the home domain. We further observe that rockfall detectors trained on multiple domains outperform single–domain trained detectors in completely unknown domains by up to 16% AP, using image data from Ceres and comet 67P. We conduct an experiment varying only image resolution on a single planetary body (Mars) to test whether the improvement was due to training on differing resolutions specifically and show that none of the improvement can be explained by this effect alone. This means that the benefits of multi–domain training mostly draw from either variations in lighting condition, differing physical appearance/backgrounds around the target of interest for generalization purposes, or both. Our findings have important applications such as machine learning–enabled science discovery in legacy and new planetary datasets. The used dataset of martian and lunar rockfalls including a detailed description is available here: https://edmond.mpdl.mpg.de/imeji/collection/DowTY91csU3jv9S2 .