Abstract

Probabilistic Models of Human Sentence Processing

Frank Keller
University of Edinburgh

Probabilistic models of human sentence processing (Jurafsky, 1996;
Corley & Crocker, 1996) offer a principled way of combining
grammatical knowledge with knowledge about the distributional
properties of linguistic structures. A number of researchers have
recently extended this approach by incorporating information-theoretic
notions into models of sentence processing (Hale, 2001; Genzel &
Charniak, 2002). These information-theoretic models rely on the
hypothesis that the amount of information conveyed by a word or
sentence predicts the processing effort associated this word or
sentence.


In this talk, we present a series of studies that test this hypothesis
using eye-tracking data. Such data provide a moment-by-moment record
of sentence processing: words that are more difficult to process cause
longer eye-fixations, or induce reverse eye-movements. We use these
data to investigate the predictions of Genzel & Charniak's (2002)
entropy rate principle. This principle states that speakers produce
text such that the entropy rate remains constant from sentence to
sentence. This makes a number of predictions: (a) entropy rate and
processing effort are correlated, (b) in connected text, processing
effort is independent of sentence position, and (c) for isolated
sentences, entropy rate (and hence processing effort) increases with
sentence position. Using a corpus of eye-tracking data, we show that
predictions (a) and (b) are borne out. We also present the results of
a reading experiment which confirms prediction (c).



Corley, S., & Crocker, M. W. (1996). Evidence for a tagging model of
human lexical category disambiguation. In Proceedings of the 18th
Annual Conference of the Cognitive Science Society, Mahwah,
NJ. Lawrence Erlbaum Associates.



Genzel, D., & Charniak, E. (2002). Entropy rate constancy in text. In
Proceedings of the 40th Annual Meeting of the Association for
Computational Linguistics, (pp. 199-206), Philadelphia.



Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic
model. In Proceedings of the 2nd Conference of the North American
Chapter of the Association for Computational Linguistics,
Pittsburgh, PA.



Jurafsky, D. (1996). A probabilistic model of lexical and syntactic
access and disambiguation. Cognitive Science, 20, 137-194.