A Semi-Automatic Evaluation Scheme: Automated Nuggetization for Manual Annotation

In this paper we describe automatic information nuggetization and its application to text comparison. More specifically, we take a close look at how machine-generated nuggets can be used to create evaluation material. A semiautomatic annotation scheme is designed to produce gold-standard data with exceptionally high inter-human agreement.