In 2002, Daniel Kahneman was awarded the Nobel Prize for Economics. He received the award for his groundbreaking work on human irrationality in collaboration with Amos Tversky in the 1970s.
In 1999, Daniel Kahneman was the lead editor of the book “Well-Being: The foundations of Hedonic Psychology.” Subsequently, Daniel Kahneman conducted several influential studies on well-being.
The aim of the book was to draw attention to hedonic or affective experiences as an important, if not the sole, contributor to human happiness. He called for a return to Bentham’s definition of a good life as a life filled with pleasure and devoid of pain a.k.a displeasure.
The book was co-edited by Norbert Schwarz and Ed Diener, who both contributed chapters to the book. These chapters make contradictory claims about the usefulness of life-satisfaction judgments as an alternative measure of a good life. Ed Diener is famous for his conception of wellbeing in terms of a positive hedonic balance (lot’s of pleasure, little pain) and high life-satisfaction. In contrast, Schwarz is known as a critic of life-satisfaction judgments. In fact, Schwarz and Strack’s contribution to the book ended with the claim that “most readers have probably concluded that there is little to be learned from self-reports of global well-being” (p. 80).
To a large part, Schwarz and Strack’s pessimistic view is based on their own studies that seemed to show that life-satisfaction judgments are influenced by transient factors such as current mood or priming effects.
“the obtained reports of SWB are subject to pronounced question-order- effects because the content of preceding questions influences the temporary accessibility of relevant information” (Schwarz & Strack, p. 79).
There is only one problem with this claim; it is only true for a few studies conducted by Schwarz and Strack. Studies by other researchers have produced much weaker and often not statistically reliable context effects (see Schimmack & Oishi, 2005, for a meta-analysis).
In fact, a recent attempt to replicate Schwarz and Strack’s results in a large sample of over 7,000 participants failed to show the effect and even found a small, but statistically significant effect in the opposite direction (ManyLabs2).
When Daniel Kahneman wrote his popular book “Thinking Fast and Slow), published in 2011, it was clear that Schwarz and Strack’s claims in the 1999 book were not representative of the broader literature on well-being. However, Chapter 9 relies exclusively on one of Schwarz and Strack’s studies that failed to replicate.
A survey of German students is one of the best examples of substitution. The survey that the young participants completed included the following two questions:
How happy are you these days?
How many dates did you have last month?
The experimenters were interested in the correlation between the two answers. Would the students who reported many dates say that they were happier than those with fewer dates?
Surprisingly, no: the correlation between the answers was about zero. Evidently, dating was not what came first to the students’ minds when they were asked to assess their happiness.
Another group of students saw the same two questions, but in reverse order:
How many dates did you have last month?
How happy are you these days?
The results this time were completely different. In this sequence, the correlation between the number of dates and reported happiness was about as high as correlations between psychological measures can get.
What happened? The explanation is straightforward, and it is a good example of substitution. Dating was apparently not the center of these students’ life (in the first survey, happiness and dating were uncorrelated), but when they were asked to think about their romantic life, they certainly had an emotional reaction. The students who had many dates were reminded of a happy aspect of their life, while those who had none were reminded of loneliness and rejection. The emotion aroused by the dating question was still on everyone’s mind when the query about general happiness came up.
Kahneman did inform his readers that he is biased against life-satisfaction judgments. Having come to the topic of well-being from the study of the mistaken memories of colonoscopies and painfully cold hands, I was naturally suspicious of global satisfaction with life as a valid measure of well-being (Kindle Locations 6796-6798). Later on, he even admits to his mistake. Life satisfaction is not a flawed measure of their experienced well-being, as I thought some years ago. It is something else entirely (Kindle Location 6911-6912).
However, he does not inform his readers about scientific evidence that these judgments are much more valid than the unrepresentative study by Schwarz and Strack suggests.
How can we explain the biased presentation of life-satisfaction judgments in Kahneman’s book. The explanation is simple. Scientists, even Nobel Laureates, are human, and humans are not always rational, which was exactly the point of Kahneman’s early work. Scientists are supposed to engage in slow information processing to use all of the available evidence and integrate it in the most systematic and objective way possible. However, scientific thinking is slow and effortful. Inevitably, scientists sometimes revert to everyday human information processing that is more error prone, but faster.
To write about life-satisfaction judgments, Kahneman could have done a literature search and retrieved all relevant studies or a meta-analysis of these studies (Schimmack & Oishi, 2005). However, a faster way to report about life-satisfaction judgments was to rely on memory, which led to the retrieval of Schwarz and Strack’s sensational finding. In this way, the reliance on a single study is a good example of substitution. What should be answered based on an objective assessment of a whole literature was answered based on a single study because it was falsely assumed that the result was representative of the literature.
For researchers like Ed Diener and his students, including myself, it has been frustrating to see that the word of an eminent Nobel Laureate has trumped scientific evidence. The recent failure to replicate Schwarz and Strack’s findings even with over 7,000 participants may help to correct the false belief that item-order effects are pervasive and that life-satisfaction judgments are invalid. Even Daniel Kahneman does not believe this anymore.
So Chapter 9 in Daniel Kahneman’s book “Thinking Fast and Slow” is as, or even more, disappointing than Chapter 4, which reported about social priming studies that failed to replicate and provide no empirical evidence for the claims made in that chapter.
Before I end, I have to make clear that my review of Chapter’s 4 and 9 should not be generalized to other chapters. I do believe that my criticism of these chapters is valid, but these chapters are not a representative sample of chapters. The scientific validity of the other chapters needs to be assessed chapter by chapter, and that takes time and effort. The reason for the focus on Chapter 9 is that I use life-satisfaction judgments in my research (which may make me biased in the opposite direction) and because the key finding featured in Chapter 9 just failed to replicate in a definitive replication study with over 7,000 participants. I think readers who bought the book might be interested to know about this replication failure.
Kahneman, Daniel. Thinking, Fast and Slow. Doubleday Canada. Kindle Edition.