Language Models as Epistemic Interfaces: Knowledge, Agents, and Their Limits

Bhuwan Dhingra

Duke University

The NLP Reading Group is excited to host Prof. Bhuwan Dhingra who will present a talk on Language Models as Epistemic Interfaces: Knowledge, Agents, and Their Limits.

Logistics

Date: Friday April 17
Time: 2PM
Location: on Google Meet, to be screencast at Mila in A14

Abstract

Language models are rapidly becoming the primary interface through which people consume information and delegate complex tasks. They are deployed in two increasingly dominant modes: as knowledge interfaces that synthesize answers from vast corpora, and as agents that interact with tools and environments over long horizons. Both modes offer remarkable capabilities, but both also obscure important epistemic signals in ways that become more consequential as these systems grow more trusted.

In this talk, I will present recent work from my lab on understanding and improving LMs along both dimensions. On the knowledge interface side, I will introduce a retrieval-free approach to knowledge attribution that enables models to cite their pretraining data, and a framework for calibrating long-form generation where correctness and confidence are treated as distributions rather than binary outcomes. On the agentic side, I will show that coding agents are surprisingly effective long-context processors – including a strong result on ARC-AGI-3 – but also that RL-trained agents develop characteristic failure modes over long horizons that aggregate benchmarks tend to hide. I will argue that both settings face a common challenge: making the reasoning and uncertainty of these systems legible to the humans who depend on them.

Speaker Bio

Bhuwan Dhingra is an Assistant Professor of Computer Science at Duke University and a Research Scientist at Apple. He has also spent time at Google DeepMind as part of the post-training team for the Gemini foundation models. His research focuses on improving the trustworthiness and efficiency of large language models for knowledge-intensive tasks. He has served as a Senior Area Chair for ACL and an Area Chair for NeurIPS, ICLR, ICML, and EMNLP. He received his bachelor’s degree from IIT Kanpur and his Ph.D. from Carnegie Mellon University. His research is supported by grants from the NSF, Amazon, Procter & Gamble, Thinking Machines Labs, and the Learning Engineering Virtual Institute. He received the Amazon Research Award in 2021.