Publications

Please see my Google Scholar page for an up-to-date list.

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, Aaron Mueller. arXiv preprint. [paper] [code]

Findings of the BabyLM Challenge: Sample-efficient Pretraining on Developmentally Plausible Corpora
Alex Warstadt*, Aaron Mueller*, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, Ryan Cotterell. Proceedings of the shared task at the Conference on Computational Natural Language Learning (CoNLL). [website] [paper]

In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax
Aaron Mueller, Albert Webson, Jackson Petty, Tal Linzen. North American Chapter of the Association for Computational Linguistics (NAACL). [paper] [code]

Function Vectors in Large Language Models
Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, David Bau. International Conference on Learning Representations (ICLR). [website] [paper] [code]

How to Plant Trees 🌳 in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases
Aaron Mueller, Tal Linzen. Association for Computational Linguistics (ACL). [paper] [code]

Meta-learning with Demonstration Retrieval for Efficient Few-shot Learning
Aaron Mueller, Kanika Narang, Lambert Mathias, Qifan Wang, Hamed Firooz. Findings of the Association for Computational Linguistics (ACL). [paper]

Language Model Acceptability Judgments Are Not Always Robust to Context (Outstanding Paper Award)
Koustuv Sinha, Jon Gauthier, Aaron Mueller, Kanishka Misra, Keren Fuentes, Roger Levy, Adina Williams. Association for Computational Linguistics (ACL). [paper]

Inverse Scaling: When Bigger Isn't Better (Featured Paper)
Ian R. McKenzie, Alexander Lyzhov, Michael Martin Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Xudong Shen, Joe Cavanagh, Andrew George Gritsevskiy, Derik Kauffman, Aaron T. Kirtland, Zhengping Zhou, Yuhui Zhang, Sicong Huang, Daniel Wurgaft, Max Weiss, Alexis Ross, Gabriel Recchia, Alisa Liu, Jiacheng Liu, Tom Tseng, Tomasz Korbak, Najoung Kim, Samuel R. Bowman, Ethan Perez. Transactions on Machine Learning Research (TMLR). [paper]

What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang, Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Jason Phang, Samuel R. Bowman. Association for Computational Linguistics (ACL). [paper]

Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models
Aaron Mueller, Robert Frank, Tal Linzen, Luheng Wang, Sebastian Schuster. Association for Computational Linguistics (ACL). [paper] [code]

Label Semantic Aware Pre-training for Few-shot Text Classification
Aaron Mueller, Jason Krone, Salvatore Romeo, Saab Mansour, Elman Mansimov, Yi Zhang, Dan Roth. Association for Computational Linguistics (ACL). [paper]

Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models
Aaron Mueller, Yu Xia, Tal Linzen. Conference on Computational Natural Language Learning (CoNLL). [paper] [code]

Bernice: A Multilingual Pre-trained Encoder for Twitter
Alexandra DeLucia, Shijie Wu, Aaron Mueller, Carlos Aguirre, Mark Dredze. Empirical Methods in Natural Language Processing (EMNLP). [paper]

Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling
Aaron Mueller, Mark Dredze. North American Chapter of the Association for Computational Linguistics (NAACL). [paper] [poster] [code] [bib]

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models
Aaron Mueller*, Matthew Finlayson*, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, Yonatan Belinkov. Association for Computational Linguistics (ACL). [paper] [code] [bib]

Decoding Methods for Neural Narrative Generation
Alexandra DeLucia*, Aaron Mueller*, Xiang Lisa Li, João Sedoc. Workshop on Generation Evaluation and Metrics (GEM), at the Association for Computational Linguistics (ACL). [paper] [code] [bib]

Demographic Representation and Collective Storytelling in the Me Too Twitter Hashtag Activism Movement
Aaron Mueller, Zach Wood-Doughty, Silvio Amir, Mark Dredze, Alicia Lynn Nobles. Proceedings of the Association for Computing Machinery (ACM) on Human-Computer Interaction (HCI), Vol. 5 (CSCW1). [paper] [bib]

Cross-Linguistic Syntactic Evaluation of Word Prediction Models
Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, Natalia Talmina, Tal Linzen. Association for Computational Linguistics (ACL). [paper] [code] [bib]

An Analysis of Massively Multilingual Neural Machine Translation for Low-Resource Languages
Aaron Mueller, Garrett Nicolai, Arya D. McCarthy, Dylan Lewis, Winston Wu, David Yarowsky. Language Resources and Evaluation Conference (LREC). [paper] [bib]

The Johns Hopkins University Bible Corpus: 1600+ Tongues for Typological Exploration
Arya D. McCarthy, Rachel Wicks, Dylan Lewis, Aaron Mueller, Winston Wu, Oliver Adams, Garrett Nicolai, Matt Post, David Yarowsky. Language Resources and Evaluation Conference (LREC). [paper] [bib]

Fine-grained Morphosyntactic Analysis and Generation Tools for More Than One Thousand Languages
Garrett Nicolai, Dylan Lewis, Arya D. McCarthy, Aaron Mueller, Winston Wu, David Yarowsky. Language Resources and Evaluation Conference (LREC). [paper] [bib]

Modeling Color Terminology Across Thousands of Languages
Arya D. McCarthy, Winston Wu, Aaron Mueller, Bill Watson, David Yarowsky. Empirical Methods in Natural Language Processing & International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). [paper] [bib]