Monday, September 26, 2011

Confirmation Bias and Positive Test Strategy

Confirmation bias is a cognitive bias in which one tends to search for, interpret, favor or remember information which confirms one's preconceived beliefs, while undervaluing or ignoring information which contradicts those beliefs.

The term was originally coined by English psychologist Peter Wason. He conducted a study which he believed demonstrated confirmation bias. Later researchers would argue that Wason did not demonstrate confirmation bias but instead showed a tendency toward "positive test strategy" (See below).

The seminal confirmation bias study was conducted at Stanford University in the late 1970's. The researchers used two groups of subjects; those that supported the death penalty and those that opposed it. The two groups both then read two studies, one seemingly supporting and the other seemingly opposing capital punishment. As predicted by the researchers, the subjects rated the study which confirmed their pre-existing beliefs as superior. In actuality, both studies were fictitious.

Positive Test Strategy
Wason's original study involved asking the subjects in the experiment to discover the rule which applied to triples of numbers. From the beginning the subjects were told that 2-4-6 fit the rule. The subjects could then give their own set of three numbers and the experimenter would tell them whether or not they confirmormed to the rule. The results showed that most people tended to first form a hypothesis (often that the rule was a sequence of ascending even numbers) and then tried to positively test it by proposing additional sequences which fit the hypothesis (such as 4-6-8 or 12-14-16). Each time the experimenter would answer positively and after a number of confirmaitons the subjects felt confident and proposed their answer (a sequence of ascending even numbers). Most often the subjects were wrong for the actual rule was simply any ascending sequence. The results showed that most people only tried to test their hypothesis by asking a set of three numbers which would prove it to be true. Very few proposed a sequence of numbers which would actually disprove their hypothesis (using the example above, simply proposing the sequence 11-13-15 would have proven the ascending even number hypothesis false). Though Wason believed this was a result of confirmation bias, later reserchers such as Joshua Klayman and Young-Won Ha would argue that it was actually a tendency to use what they called Positive Test Strategy. In their view, this tendency is simply a preference to use test which seek to verify a hypothesis as opposed to disprove it. "Klayman and Ha (1987) argued that a major problem in testing hypothesis is deciding whether, on average, conducting +Htest or -Htest will be most informative. They suggested that people tend to use a simple approach: Select the strategy that is likely to have the greatest impact on your belief in the current hypothesis"(2)

I think the best way to avoid confirmation bias is to periodically perform a self diagnostic test where one introspectively questions ones attachments to a particular ideology, pattern of thinking, etc. I think that sometimes we hold onto old ways of thinking simply because we do not want to be proven wrong. Other times, the problem seems much deeper, perhaps the way we see the world rest upon a conceptual framework which if proven wrong would cause us anxiety and make us feel uncertain. Perhaps this is not just limited to how we see the world but also includes aspects of self identity. Consciously or unconsciously we no doubt want to confirm evidence that supports our sense of who we our or at least how we see ourselves and downplay contradictory information. Regardless of the reasons periodic introspective questioning is important. Also, as positive test strategy suggest, the methods we go about finding the truth should include both positive and negative hypothesis testing.

Wikipedia: Confirmation Bias
The Psychology of Bias: Understanding and Eliminating Bias in Investigations

No comments:

Post a Comment