3

Sampling from Your Language Model One Byte at a Time
Tokenization is used almost universally by modern language models, enabling efficient text representation using multi-byte or …
Jonathan Hayase, Alisa Liu, Noah A. Smith, Sewoong Oh
Paper BibTeX Code
Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
Modern tokenizers employ deterministic algorithms to map text into a single “canonical” token sequence, yet the same string …
Brian Siyuan Zheng, Alisa Liu, Orevaoghene Ahia, Jonathan Hayase, Yejin Choi, Noah A. Smith
Paper BibTeX Code
Tuning Language Models by Proxy
We develop an algorithm for “tuning” language models at decoding-time!
Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, Noah A. Smith
Paper BibTeX Code