Alexander "Sasha" Rush won a Best Paper Award at the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). "How many data points is a prompt worth?" won Outstanding Short Paper. The paper is part of a collaboration with Teven Le Scao and Rush's work with Hugging Face.
Rush's research in this vein was earlier celebrated with a Best Demo Award at the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) for their paper "
Transformers: State-of-the-Art Natural Language Processing."
In "How many data points is a prompt worth?," Rush and Le Scao explore how "when fine-tuning pretrained models for classification, researchers either use a generic model head or a task-specific prompt for prediction. Proponents of prompting have argued that prompts provide a method for injecting task- specific guidance, which is beneficial in low- data regimes. We aim to quantify this benefit through rigorous testing of prompts in a fair setting: comparing prompted and head-based fine-tuning in equal conditions across many tasks and data sizes. By controlling for many sources of advantage, we find that prompting does indeed provide a benefit, and that this ben- efit can be quantified per task. Results show that prompting is often worth 100s of data points on average across classification tasks."
Read more about Rush's other recent award-winning research.