1

SuperBPE: Space Travel for Language Models
The assumption across nearly all language model (LM) tokenization schemes is that tokens should be subwords, i.e., contained within …
Alisa Liu*, Jonathan Hayase*, Sewoong Oh, Noah A. Smith, Yejin Choi
Paper BibTeX Code Blog HF Collection
Tulu 3: Pushing Frontiers in Open Language Model Post-Training
Language model post-training is applied to refine behaviors and unlock new skills across a wide range of recent language models, but …
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, Hannaneh Hajishirzi
Paper BibTeX Code HF Collection
Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models
Despite their wide adoption, the biases and unintended behaviors of language models remain poorly understood. In this paper, we …
Hila Gonen, Terra Blevins, Alisa Liu, Luke Zettlemoyer, Noah A. Smith
Paper BibTeX Dataset
LlamaPIE: Proactive In-Ear Conversation Assistants
We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise …
Tuochao Chen, Nicholas Batchelder, Alisa Liu, Noah Smith, Shyamnath Gollakota
Paper BibTeX
Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?
Despite the general capabilities of large pretrained language models, they consistently benefit from further adaptation to better …
Jonathan Hayase*, Alisa Liu*, Yejin Choi, Sewoong Oh, Noah A. Smith
Paper BibTeX Code
We're Afraid Language Models Aren't Modeling Ambiguity
We build a benchmark to evaluate LM understanding of ambiguity, which is an intrinsic feature of language, and find that the task remains extremely challenging, including for GPT-4
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, Yejin Choi
Paper BibTeX Code Dataset
That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?
The translation of ambiguous text presents a challenge for translation systems, as it requires using the surrounding context to …
Jaechan Lee, Alisa Liu, Orevaoghene Ahia, Hila Gonen, Noah A. Smith
Paper BibTeX Code
Inverse Scaling: When Bigger Isn't Better
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale …
Ian R. McKenzie, 18 others, Alisa Liu, Jiacheng Liu, Tom Tseng, Tomasz Korbak, Najoung Kim, Samuel R. Bowman, Ethan Perez
Paper BibTeX Code
How Language Model Hallucinations Can Snowball
A major risk of using language models in practical applications is their tendency to hallucinate incorrect statements. Hallucinations …
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, Noah A. Smith
Paper BibTeX Code
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Large “instruction-tuned” language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to …
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi
Paper BibTeX Code