A Critique of Hagoort et al., Integration of Word Meaning and World Knowledge in Language Comprehension

Kira McManus

Writer’s comment: When I selected an article to review for my Psychology 132: Language and Cognition class, I selected the topic which sounded the most interesting.  But upon reading the article, I discovered that interesting meant complicated.  This paper contained processes which were difficult for me to understand, and vocabulary which I had never seen before.  I forced myself to spend countless hours in the library reading research articles and attempting to weed information from them.  I even went to office hours, which is something I never do.  The more I researched for the paper, the more I wanted to acquire knowledge.  It was through this research that I discovered that the process by which language is brought into the brain is fascinating, not to mention the different ways we can study these processes. As a result of my hard work I was able to gain so much interest into the field of cognitive psychology.  Although this paper was one of the most difficult papers I have written at Davis, it has been by far the most rewarding.
-Kira McManus

Instructor's comment: Language and Cognition (Psychology 132) provides an introduction to the cognitive processes and brain mechanisms involved in language comprehension and production. One of the students’ assignments was to write a review—in the format of a scientific paper with Introduction, Methods, Results, and Discussion sections—of a seminal study of language processes in the brain as published in the journal Science.  Kira chose a topic  that has puzzled philosophers for centuries: Can we distinguish meaning information (semantics) from our knowledge of the world? For example, whereas the following statement is semantically correct, it is not true according to our knowledge of the world: “The president of the United States is a widower.” Sentences can also be truly semantically incorrect: “The president of the United States is a jellyfish”. Questions of the separability in the processing and representation of different types of information are rooted in psychological debates on the degree of modularity or interactivity of language with other cognitive processes. Modular views have predicted that humans first process the semantic information and then assign a true or false value to a statement. However, Kira’s clear review of the paper by Hagoort and colleagues (2004) indicates that even though the brain can keep track of these different types of information, it processes semantic information and knowledge of the world in parallel, and without delay.
-Tamara Swaab, Psychology, Center for Mind and Brain

Various theories have been proposed to explain how language is integrated into the brain.  One prominent theory argues for a “nonoverlapping two-step interpretation procedure in which first the meaning of the sentence is determined, and only then is its meaning verified in relation to our knowledge of the world” (Hagoort, P., Hald, L., Bastiaansen, M., & Petersson, K.M., 2004, p. 440).  This interpretation can be likened to other modular theories in which the components of language can be broken down into separate sections or groups, as opposed to interactive theories in language in which everything works together in a simultaneous process. In terms of modularity, it has been proposed that a distinction “can be made between the facts of the world and the facts of the words of our language, including their meaning” (Hagoort et al., 2004, p. 438). Others disagree with this remark and claim that no division can be made between pragmatics, or world knowledge, and semantics, or word knowledge.  The study performed by Hagoort, Hald, Bastiaansen, and Petersson (2004) attempts to settle this particular debate between interactivity and modularity by using neurophysiological methods as well as the theory of a distinction between world and word knowledge.

 

Methods:

To begin with, Hagoort et al. (2004) devised multiple sentences, each of which had three variations: “The Dutch trains are yellow/white/sour and very crowded” (Hagoort et al., 2004, p. 439).  In Holland, it is known that Dutch trains are yellow.  Therefore the first of these three sentences is true.  The other two sentences both make violations, the sentence with the word “white” being a world knowledge violation (trains in Holland are yellow and not white), while  the sentence with the word “sour” is a semantic violation (trains are not generally related to flavor). With these three types of sentences, Hagoort et al. (2004) used three different measurement techniques to evaluate the participants’ brain reactions. They used an electroencephalogram (EEG) to examine event-related potentials as well as oscillatory brain activity, and functional magnetic resonance imaging (fMRI) to identify the active brain areas during sentence processing.

Hagoort et al. (2004) decided to examine a distinctive event-related potential known as the N400.  A negative spike that typically reaches its peak at 400 milliseconds, the N400 was first discovered by Marta Kutas and Steven A. Hillyard (1980).  As discussed in the Kutas and Hillyard article, the negative spike was brought on by a semantic violation. In this experiment, the N400 spike was expected with the semantic violation sentence (“sour”).  The two-step process of language integration predicts that the onset of the brain’s reaction to the critical word occurs later in the pragmatic violation sentence than that same word in the semantic violation sentence.  The other theory, however, predicts no difference between the N400 of the semantic violation and that of the pragmatic violation. 

As for the brain oscillations, Hagoort et al. (2004) looked at two key brain frequencies, the gamma oscillation and the theta oscillation.  Gamma oscillations (30-70 Hz) “are suggested to play an important role in the integration of activity in both local and distributed neural networks” (Hagoort et al., 2004, p. 440), while theta oscillations (4-7 Hz) have to do with “both episodic and working memory tasks” (Hagoort et al., 2004, p.440).  Hagoort et al. was attempting to discover in the fMRI data those areas of the brain most active during the violation sentence processing.

In other similar studies, such as the one performed by Duffy and Keir (2004), different methods were employed in place of the neurophysiological approach of Hagoort et al. (2004).  In the Duffy and Keir experiment, eye movement was measured in response to reading about gender stereotypes.  Although the neurophysiological methods in the Hagoort et al. study were highly effective, it would also therefore be possible to perform a study like theirs using eye movement tracking.

 

Results:

In the study by Hagoort et al., the peak latency and onset of N400 for the two different kinds of violations were actually identical.  The amplitude was also very similar, but slightly larger for the semantic violation.  In the brain oscillation observations, a distinct gamma peak occurred during the pragmatic violation, but did not occur during the semantic violation.  On the other hand, a theta peak appeared which was much larger for the semantic violation than the pragmatic violation. As for the fMRI data, Hagoort et al. (2004) showed that the left prefrontal cortex, mostly Brodmann’s areas 45 and 47, were highly active when both types of violation sentences were read.  The left prefrontal cortex is associated with sentence processing, being one of the main N400 generators (Frishkoff, G.A., Tucker, D.M., Davey, C., and Scherg, M., 2004).

 

Discussion:

Hagoort et al.’s N400 results disprove the two-step theory of language integration. The onset of the N400 was identical in both the world knowledge sentence as well as the word knowledge sentence, showing that a sentence requiring the integration of external knowledge does not result in a signal delay. A similar study by van Berkum, Hagoort, and Brown (1999) discusses the difference between word integration on a sentence level and word integration on a discourse-level. This study also used the N400 effect, but in an attempt to figure out whether it takes longer to recognize an anomalous word in context or if it takes the same amount of time as a sentence without any given background.  Van Berkum et al.’s findings were similar to those of Hagoort et al., demonstrating that sentences involving prior knowledge (in the case of Hagoort et al., world knowledge) do not require more time to process than sentences that do not.  With these two studies we are able to dismiss the two-step theory.  Instead, the brain seems to access everything at the same time. The fMRI portion of the Hagoort et al. study also helps to show that the brain uses this simultaneous process.  

In fact, the fMRI portion of the study offered some entirely new findings.  Before this study, the left inferior prefrontal cortex was thought to be mainly involved in semantic processing.  We now know that the LIPC is also involved in pragmatic processing. In a brain lesion study conducted in 2004 by Dronkers et al., Broadmann’s area 47 (which is one of the areas examined in the Hagoort et al. (2004) study) as well as multiple areas surrounding Broca’s area were discovered to exert a large impact on the comprehension of language.  Therefore it would make sense that this area would be activated during the comprehension of sentence violations.  The fact that this one area ignited during the world knowledge violation sentences shows that perhaps the violation of a word definition and the violation of a world knowledge definition cannot be separated. 

The last part of the Hagoort et al. study measured the brain oscillations.  The study claims that these results “suggest that the brain keeps a record of what makes a sentence hard to interpret” (Hagoort et al, 2004, p. 440).  This conclusion is based on the theta and gamma oscillations, but the evidence provided in the article is only indirectly conclusive.  It provides, instead, the basis for another experiment.  Simply because the theta oscillations seem to coincide with working memory still does not prove that the theta oscillation in this particular experiment was brought on by the memory’s activation.  Although this is an interesting conclusion, this one experiment does not prove it.

Even though “Integration of word meaning and world knowledge in language comprehension” jumps to conclusions about the significance of brain oscillations, overall it presents a highly effective experiment.  In a three-page article, Hagoort et al. (2004) not only disproves the two-step theory, but shows that the left inferior prefrontal cortex is highly involved in the integration of words and knowledge in the brain. 

 

References:

Dronkers, N.F., Wilkins, D.P., Van Valin, Jr., R.D., Redfern, B.B., & Jaeger, J.J. (2004) Lesion analysis of the brain areas involved in language comprehension. Cognition, 92, 145-77.

Duffy, S.A. & Keir, J.A. (2004). Violating stereotypes: Eye movements and comprehension processes when text conflicts with world knowledge. Memory & Cognition, 32(4), 551-9.

Frishkoff, G.A., Tucker, D.M., Davey, C., & Scherg, M. (2004). Frontal and posterior sources of event-related potentials in semantic comprehension. Cognitive Brain Research, 20, 329-54.

Hagoort P., Hald, L., Bastiaansen, M. & Petersson, K.M. (2004). Integration of word meaning and world knowledge in language comprehension. Science. 304, 438-41.

Kutas, M. & Hillyard, S.A. (1980). Event-related potentials to semantically inappropriate and surprisingly large words. Biological Psychology, 11, 99-116.

van Berkum, J.J.A., Hagoort, P., & Brown, C.M. (1999). Semantic integration in sentences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience, 11(6), 657-71.