The Experimental Philosophy Blog

Philosophy Meets Empirical Research

Menu
  • Home
  • About
  • Guidelines for Comments
  • Labs and Organizations
  • Resources
Menu

Tag: Replication

Priming Effects Are Fake, but Framing Effects Are Real

Posted on May 23, 2025June 6, 2025 by Joshua Knobe

A few decades ago, it was pretty common to mush together priming effects and framing effects and see them as two closely connected parts of a single Bigger Truth about the human mind. Of course, everyone understood that the effects themselves were a bit different, but one common view was that they were providing evidence for the same larger picture. That larger picture said: People’s judgments are radically unstable, easily pushed around by subtle and almost unnoticeable factors.

Things have changed so much since then. Priming research in social psychology has experienced a series of truly spectacular replication failures, while research on framing effects continues to look very solid. In light of this change, we should rethink our understanding of what framing effects show about human cognition. We shouldn’t see them as part of a larger picture that also includes priming. We need an understanding of framing that allows us to situate it within a larger picture, according to which priming effects are not real.

The priming literature seemed to be showing that people’s judgment and decision-making are highly unstable and can be easily shifted around by small manipulations of the external situation. The thought was that if you just happen to be holding a hot coffee, or sitting at a dirty desk, or in a room that includes a picture of dollar bills, your whole way of thinking about things will be shifted in some fundamental respect. For example, you will end up making deeply different moral judgments.

The key lesson of more recent research is simple: these priming effects do not occur. More generally, we cannot shift people’s moral judgments around in some radical way just by making subtle changes in their situation. Your moral judgments will not shift around completely if you are seated at a dirty desk. That is not how the human mind works.

Okay, with all of that in mind, let’s rethink framing effects. For concreteness, we can focus on a famous study from Tversky and Kahneman (1981). In this study, participants were randomly assigned to one of two conditions. Participants in the gain framing condition read the following case:

A disease is expected to kill 600 people. You can choose between two options:

  • If you choose the first option, 200 people will be saved.
  • If you choose the second option, there is one-third probability that 600 people will be saved and a two-thirds probability that 0 people will be saved.

Meanwhile, participants in the loss framing condition read:

A disease is expected to kill 600 people. You can choose between two options:

  • If you choose the first option, 400 people will die.
  • If you choose the second option, there is one-third probability that 0 people will die and a two-thirds probability that 600 people will die.

Clearly, the two descriptions are logically equivalent, but they tend to yield very different responses. Participants tend to be risk-averse in the first case, risk-seeking in the second.

During the heyday of priming research, many of us thought that this sort of effect should be understood within a larger picture of the mind that also included priming. Basically, the idea was something like this: “People’s judgments about a case can be shifted around but all sorts of little things, including everything from the decor in the room to the precise words used to describe it.” But in light of everything we know now, we need to revisit this view. Framing effects are very real, but that larger picture seems to be mistaken. We need to understand framing effects within a larger picture of the mind, according to which people’s judgments don’t just shift around randomly as a result of all sorts of little factors.

I’d be very open to different views about what the right picture is, but just as a first step in this direction, let’s consider a picture that emerges not from social psychology but rather from very traditional work in philosophy. This picture says that people often have a collection of different intuitions that are mutually inconsistent. These intuitions need not be unstable in any way. It might be that each individual intuition is completely stable; it’s just that the different intuitions contradict each other.

To illustrate, consider intuitions about free will. I might find myself having the following three intuitions: (a) All human behavior is completely explained by genes and environment, (b) If a person’s behavior is completely explained by genes and environment, that person’s behavior is not performed with free will, (c) Some human behaviors are performed with free will. These three intuitions are mutually inconsistent, so they cannot all be right. However, this does not mean that people’s free will intuitions have to be unstable in any way.

On the contrary, a single individual could easily have all three intuitions at the same time. For example, as a philosopher, I might start out a paper by explaining that each of these three claims seems intuitively to be true, that they are mutually inconsistent and hence cannot all be right, and that we therefore face an interesting philosophical problem. Alternatively, someone might simply have each of these three intuitions, but without noticing that they contradict each other. In such a case, the person would be failing to notice something important, but that would not mean that the person’s intuitions were unstable. Each of the three intuitions might be perfectly stable; it’s just that the three intuitions are not consistent.

Some philosophical problems seem to have very much the structure we see in framing effects. Consider the philosophical problem of moral luck. The problem starts with three intuitions: (a) An agent who doesn’t bring about any bad outcomes deserves relatively little blame, (b) An agent who performs the exact same behavior but who ends up bringing about a bad outcome deserves a lot of blame, (c) If the agent performs the exact same behavior in two cases and the only difference is in the outcome that ends up occurring, that difference by itself cannot be relevant to how much blame the agent deserves. I myself have all three of these intuitions. Since the intuitions are mutually inconsistent, they cannot all be right, but that does not mean that my intuitions are unstable. Each of the three intuitions is completely stable and emerges in all situations; it’s just that the three intuitions are in tension with each other.

Let’s now return to framing effects. In the days when it seemed like priming was real, I totally see why researchers would think that framing was a lot like priming. But in light of subsequent studies, maybe we should see it in a completely different way. Framing does not involve people’s judgments being unstable; it instead involves people having different intuitions that are mutually inconsistent.

Take the example described above. Looking at that example, I have the following three intuitions: (a) The correct answer in the first case is to take the non-risky option, (b) The correct answer in the second case is to take the risky option, and (c) It cannot possibly be the case that the correct answer in the first case is different from the correct answer in the second case. These three intuitions are mutually inconsistent, so they cannot all be right. However, each individual intuition can be perfectly stable. In fact, thinking about the problem right now, I find myself having all three intuitions at the same time.

Turning the traditional view about framing effects upside down, one might even see framing effects as an extreme case of stability. Just as we continue to experience a visual illusion even when we know that it is illusory, we continue to have the inconsistent intuitions that together constitute a framing effect even when we know that they cannot all be right.

[I discuss this issue in this paper, but please feel free to respond to this blog post even if you haven’t looked at the full paper.]

Teaching Experimental Philosophy to Beginners (Part 1)

Posted on April 29, 2024January 3, 2025 by Alexander Max Bauer

This term at the University of Oldenburg, Stephan Kornmesser and I are teaching a course for master’s students who had no previous contact with x phi.1 We decided to try a hands-on approach rather than just discussing results, debates, and ideas from the field. For this purpose, we divided the course into two parts. In the first half, loosely based on Kornmesser et al. (forthcoming), we introduced some basics of experimental design and statistical analysis. Also, as a paradigmatic example of an early x phi study, we read Knobe (2003). In my opinion, this paper has the advantage of being both short and accessible; the experimental design is simple and the data collected (primarily the nominal yes-or-no responses) can be analyzed quite straightforwardly.

Thereafter, the course was divided into three groups,2 and students were instructed to replicate Knobe’s first study step by step. First, we reconstructed the questionnaire, using a German translation of the vignette (taken from Knobe 2014). Everyone probably knows the vignette, but to refresh your memory, here is the original once again (variations between Harm and Help Condition are indicated by square brackets):

The vice-president of a company went to the chairman of the board and said, “We are thinking of starting a new program. It will help us increase profits, but [and] it will also harm [help] the environment.”

The chairman of the board answered, “I don’t care at all about harming [helping] the environment. I just want to make as much profit as I can. Let’s start the new program.”

They started the new program. Sure enough, the environment was harmed [helped].

Knobe (2003, 190)

Afterwards, students used the questionnaire to ask people on campus whether the chairman brought about the side effect intentionally. Finally, we calculated and interpreted χ2 tests for each group. To foster an understanding of how the χ2 test actually works, we calculated them by hand one step at a time instead of using software.

As can be seen in the picture below, showing the results from Group 1, Knobe’s original findings were perfectly replicated.3 In the Harm Condition, most subjects said that the chairman brought about the side effect intentionally, while in the Help Condition, most subjects said that he did not.4

Replication of Study 1 from Knobe (2003) by Student Group 1

As did Knobe (2003), people were also asked how much praise or blame the chairman deserves. Using the results from Group 1 again as an example, shown in the picture below, the mean of ascribed blame was 4.90 (SD = 1.26), while the mean of ascribed praise was 2.32 (SD = 1.63), which also fits in very well with Knobe’s results.5

Replication of Study 1 from Knobe (2003) by Student Group 1

Where do we go from here? In the second half of our course, the three groups are encouraged to develop their own research questions based on their philosophical interests. They will learn how to design an online survey and set one up themselves. Luckily, we have got a grant for research-based learning instructional projects from the university’s initiative forschen@studium,6 which will be used to recruit subjects from an online panel provider. Hence, learning panel integration will also be on our schedule. This is followed by guided data analysis and interpretation. Finally, the groups present their results to each other and document them in term papers. This means that, in the end, an entire research process is experienced.

Literature

De Cruz, Helen (2019): “Unconventional Teaching Ideas That Work. Teaching Experimental Philosophy to Undergraduate Students,” The Philosophers’ Cocoon, https://philosopherscocoon.typepad.com/blog/2019/02/unconventional-teaching-ideas-that-work-teaching-experimental-philosophy-to-undergraduate-students-h.html.

Kornmesser, Stephan, Alexander Max Bauer, Mark Alfano, Aurélien Allard, Lucien Baumgartner, Florian Cova, Paul Engelhardt, Eugen Fischer, Henrike Meyer, Kevin Reuter, Justin Sytsma, Kyle Thompson, and Marc Wyszynski (forthcoming): Experimental Philosophy for Beginners. A Gentle Introduction to Methods and Tools, Cham: Springer. (Link)

Knobe, Joshua (2003): “Intentional Action and Side Effects in Ordinary Language,” Analysis 63 (3), 190–194. (Link)

Knobe, Joshua (2014): “Absichtliches Handeln und Nebeneffekte in der Alltagssprache,” translated by Jürgen Schröder, in: Thomas Grundmann, Joachim Horvath, and Jens Kipper (eds.): Die Experimentelle Philosophie in der Diskussion, Berlin: Suhrkamp, 96–101. (Link)

Endnotes

  1. If you are interested in teaching x phi to beginners, you should also take a look at De Cruz (2019). In her blog post, she describes a teaching approach to third-year undergraduates at Oxford Brookes University. ↩︎
  2. Of course, the great students of our course deserve credit! Group 1: Johannes Bavendiek, Marvin Jonas Laesecke, and Aileen Wiechmann; Group 2: Rebecca Kratzer, Frederike Lüttich, and Jule Rüterbories; Group 3: Bastian Göbbels, Finn Ove Gronotte, Marina Hinkel, and Riduan Schwarz. ↩︎
  3. Data and materials can be found at https://github.com/alephmembeth/course-x-phi-2024. ↩︎
  4. χ2(1, 54) = 27.865, p < 0.001. For comparison: Knobe (2003, 192) reports χ2(1, 78) = 27.2, p < 0.001. ↩︎
  5. t(54) = 6.43, p < 0.001. For comparison: Knobe (2003, 193) reports – pooled for both of his studies – a mean of 4.8 in the harm condition and of 1.4 in the help condition; t(120) = 8.4, p < 0.001. ↩︎
  6. See https://uol.de/en/forschen-at-studium. ↩︎

Search

Categories

Tags

Agency Artificial Intelligence Basic Needs Behavior Beliefs Bias Bioethics Blame Causation Cognitive Science Consciousness Corpus Analysis Cross-Cultural Research Decisions Determinism Distributive Justice Emotions Essentialism Expertise Folk Morality Framing Free Will Gender Intention Intuition Jurisprudence Knowledge Large Language Models Moral Psychology Norms Pejoratives Problem of Evil Psycholinguistics Rationality Reasoning Replication Responsibility Self Side-Effect Effect Slurs Truth Valence Values Virtue Well-Being

Recent Posts

  • Call: “The Armchair on Trial”
  • Call: “Interdisciplinary Perspectives on Linguistic Justice”
  • Call: “6th European Experimental Philosophy Conference”
  • Talk: “Ethics in the Lab” (Pascale Willemsen)
  • Talk: “Normality and Norms” (Josh Knobe)

Recent Comments

  1. Nova Praxis on The Folk Concept of ArtJuly 11, 2025

    This article highlights an important point: everyday people don’t rely on rigid definitions to determine what qualifies as art. They’re…

  2. Koen Smets on Priming Effects Are Fake, but Framing Effects Are RealMay 27, 2025

    That is indeed exactly the question I have as well. I operationalize it as having de facto contradicting intuitions, in…

  3. Joshua Knobe on Priming Effects Are Fake, but Framing Effects Are RealMay 24, 2025

    Hi Koen, Thanks once again. This idea brings up all sorts of fascinating questions, but for the purposes of the…

  4. Koen Smets on Priming Effects Are Fake, but Framing Effects Are RealMay 24, 2025

    Great! In the meantime I thought of another potentially interesting example of framing—Arnold Kling’s Three Languages of Politics. Just about…

  5. Joshua Knobe on Priming Effects Are Fake, but Framing Effects Are RealMay 23, 2025

    Thanks Koen! This is all super helpful.

Archives

  • February 2026
  • January 2026
  • December 2025
  • November 2025
  • October 2025
  • September 2025
  • August 2025
  • July 2025
  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org

Imprint • Disclaimer • Privacy Statement • Cookie Policy

© 2024 The Experimental Philosophy Blog
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
  • Manage options
  • Manage services
  • Manage {vendor_count} vendors
  • Read more about these purposes
View Preferences
  • {title}
  • {title}
  • {title}