Evaluating Induced CCG Parsers on Grounded Semantic Parsing
Yonatan Bisk, Siva Reddy, John Blitzer, J. Hockenmaier, Mark Steedman
EMNLP
Abstract
We compare the effectiveness of four different syntactic CCG parsers for a semantic slot-filling task to explore how much syntactic supervision is required for downstream semantic analysis. This extrinsic, task-based evaluation provides a unique window to explore the strengths and weaknesses of semantics captured by unsupervised grammar induction systems. We release a new Freebase semantic parsing dataset called SPADES (Semantic PArsing of DEclarative Sentences) containing 93K cloze-style questions paired with answers. We evaluate all our models on this dataset. Our code and data are available at this https URL