In-Context Algebra

Eric Todd

Northeastern University

The NLP Reading Group is excited to host Eric Todd who will be presenting his work In-Context Algebra.

Logistics

Date: Friday March 27
Time: 2PM
Location: on Google Meet, to be screencast at Mila in A14

Abstract

We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning is determined only through their interactions in-context. While prior work has studied transformers in settings where the answer relies on fixed parametric or geometric information encoded in token embeddings, we devise a new in-context reasoning task where the assignment of tokens to specific algebraic elements varies from one sequence to another. Despite this challenging setup, transformers achieve near-perfect accuracy on the task and even generalize to unseen groups. We develop targeted data distributions to create causal tests of a set of hypothesized mechanisms, and we isolate three mechanisms models consistently learn: commutative copying where a dedicated head copies answers, identity element recognition that distinguishes identity-containing facts, and closure-based cancellation that tracks group membership to constrain valid answers. Our findings show that the kinds of reasoning strategies learned by transformers are dependent on the task structure and that models can develop symbolic reasoning mechanisms when trained to reason in-context about variables whose meanings are not fixed.

Speaker Bio

Eric Todd is a 4th-year PhD student at Northeastern University, advised by David Bau. He received his BS in Applied and Computational Mathematics from Brigham Young University. His research focuses on in-context learning, interpretability, and causal abstraction. He is particularly interested in understanding the internal algorithms and representations that language models use to understand their context for problem solving.