Pseudo Ground Truth Segmentation Mask to Improve Video Prediction Quality

Video prediction to foresee future events is an extremely difficult job since it involves spatial feature extraction and temporal sequential analysis. We identified that the semantic information is actually crucial to prediction, and proposed using “pseudo ground truth” segmentation masks which are generated automatically in real time and add them to the input layers as extra information to predict future frames. Experiments conducted on our self-defined network demonstrated drastically higher quality predictions are achieved when compared with other state-of-the-art direct video prediction models.