Online Handwritten Diagram Recognition with Graph Attention Networks.

Handwritten text recognition has been extensively researched over decades and achieved extraordinary success in recent years. However, handwritten diagram recognition is still a challenging task because of the complex 2D structure and writing style variation. This paper presents a general framework for online handwritten diagram recognition based on graph attention networks (GAT). We model each diagram as a graph in which nodes represent strokes and edges represent the relationships between strokes. Then, we learn GAT models to classify graph nodes taking both stroke features and the relationships between strokes into consideration. To better exploit the spatial and temporal relationships, we enhance the original GAT model with a novel attention mechanism. Experiments on two online handwritten flowchart datasets and a finite automata dataset show that our method consistently outperforms previous methods and achieves the state-of-the-art performance.