1

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

Despite the general capabilities of large pretrained language models, they consistently benefit from further adaptation to better …

Jonathan Hayase*, Alisa Liu*, Yejin Choi, Sewoong Oh, Noah A. Smith

We're Afraid Language Models Aren't Modeling Ambiguity

We build a benchmark to evaluate LM understanding of ambiguity, which is an intrinsic feature of language, and find that the task remains extremely challenging, including for GPT-4

Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, Yejin Choi

That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?

The translation of ambiguous text presents a challenge for translation systems, as it requires using the surrounding context to …

Jaechan Lee, Alisa Liu, Orevaoghene Ahia, Hila Gonen, Noah A. Smith

Inverse Scaling: When Bigger Isn't Better

Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale …

Ian R. McKenzie, 18 others, Alisa Liu, Jiacheng Liu, Tom Tseng, Tomasz Korbak, Najoung Kim, Samuel R. Bowman, Ethan Perez

How Language Model Hallucinations Can Snowball

A major risk of using language models in practical applications is their tendency to hallucinate incorrect statements. Hallucinations …

Muru Zhang, Ofir Press, William Merrill, Alisa Liu, Noah A. Smith

Self-Instruct: Aligning Language Models with Self-Generated Instructions

Large “instruction-tuned” language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to …

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi

Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts

Using expert and anti-expert LMs to rewrite toxic text for safety

Skyler Hallinan, Alisa Liu, Yejin Choi, Maarten Sap

WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation

We introduce a paradigm for dataset creation based on human and machine collaboration, and demonstrate its empirical effectiveness for collecting a new large-scale NLI dataset

Alisa Liu, Swabha Swayamdipta, Noah A. Smith, Yejin Choi

Generated Knowledge Prompting for Commonsense Reasoning

Prompting GPT-3 to generate relevant background knowledge improves performance on a variety of commonsense reasoning tasks

Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi

DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts

Steering open-ended text generation toward desired or away from undesired attributes, using expert and anti-expert language models

Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi