Plug and Play Language Model: A simple baseline for controlled language generation

Large transformer-based generative models (e.g. GPT-2; 1.5B parameters) trained on a huge corpus (e.g. 40GB of text) have shown unparalleled language generation ability. While these models are powerful, fine-grained control of attributes of the generated language (e.g. gradually switching topic or sentiment) is difficult without modifying the model architecture to allow extra attribute inputs, or fine-tuning with attribute-specific data. Both would entirely change the original generative function, which, if done poorly, cannot be undone; not to mention the cost of retraining. We instead propose the Plug and Play Language Model for controlled language generation that consists of plugging in simple bag-of-words or one-layer classifiers as attribute controllers, and making updates in the activation space, without changing any model parameters. Such a control scheme provides vast flexibility and allows full recovery of the original generative function.The results demonstrate fine-grained control over a range of topics and sentiment styles, as well as the ability to detoxify generated texts. Our experiments, including human evaluation studies, show that text generated via this control scheme is aligned with desired attributes, while retaining fluency.