Using Integrated Gradients to explain Linguistic Acceptability learnt by BERT.

BERT has been a breakthrough in language understanding by leveraging the multi-head self-attention mechanism in its architecture. To the best of our knowledge this work is the first to leverage Layer Integrated Gradients Attribution Scores (LIGAS) to explain the Linguistic Acceptability criteria that are learnt by BERT on the Corpus of Linguistic Acceptability (CoLA) benchmark dataset. Our experiments on 5 different categories of sentences lead to the following interesting findings: 1) LIGAS for Linguistically Acceptable (LA) sentences are significantly smaller in comparison to Linguistically Unacceptable (LUA) sentences, 2) There are specific subtrees of the Constituency Parse Tree (CPT) for LA and LUA sentences which contribute larger LIGAS, 3) Across the different categories of sentences we observed around 88% to 100% of the Correctly classified sentences had positive LIGAS, indicating a strong positive relationship to the prediction confidence of the model, and 4) Around 57% of the Misclassified sentences had positive LIGAS, which we believe can become correctly classified sentences if the LIGAS are parameterized in the loss function of the model.