EyeSeg: Fast and Efficient Few-Shot Semantic Segmentation

Semantic segmentation is a key component in eye- and gaze- tracking for virtual reality (VR) and augmented reality (AR) applications. While it is a well-studied computer vision problem, most state-of-the-art models require large amounts of labeled data, which is limited in this specific domain. An additional consideration in eye tracking is the capacity for real-time predictions, necessary for responsive AR/VR interfaces. In this work, we propose EyeSeg, an encoder-decoder architecture designed for accurate pixel-wise few-shot semantic segmentation with limited annotated data. We report results from the OpenEDS2020 Challenge, yielding a 94.5% mean Intersection Over Union (mIOU) score, which is a 10.5% score increase over the baseline approach. The experimental results demonstrate state-of-the-art performance while preserving a low latency framework. Source code is available: http://www.cs.utsa.edu/~fernandez/segmentation.html.