Reviewing Natural Language Processing Research

This tutorial will cover the theory and practice of reviewing research in natural language processing. Heavy reviewing burdens on natural language processing researchers have made it clear that our community needs to increase the size of our pool of potential reviewers. Simultaneously, notable false negatives---rejection by our conferences of work that was later shown to be tremendously important after acceptance by other conferenceshave raised awareness of the fact that our reviewing practices leave something to be desired. We do not often talk about false positives with respect to conference papers, but leaders in the field have noted that we seem to have a publication bias towards papers that report high performance, with perhaps not much else of interest in them. It need not be this way. Reviewing is a learnable skill, and you will learn it here via lectures and a considerable amount of hands-on practice.