Heterogeneous Multi-task Learning on Non-overlapping Datasets for Facial Landmark Detection

We propose a heterogeneous multi-task learning framework on non-overlapping datasets, where each sample has only part of the labels and the size of each dataset is different. In particular, we propose two batch sampling strategies for stochastic gradient descent to learn shared CNN representation. First one sets same number of iteration on each dataset while the latter sets same batch size ratio of one task to another. We evaluate the proposed framework by learning the facial expression recognition task and facial landmark detection task. The learned network is memory efficient and able to carry out multiple tasks for one feed forward with the shared CNN. In addition, we show that the learned network achieve more robust facial landmark detection under large variation which appears in the heterogeneous dataset, though the dataset does not include landmark labels. We also investigate the effect of weights on each cost function and batch size ratio of one task to another.