You Should Downvote Contrarian Anecdotes

In discussions on the findings of a piece of research, a handful of contrarian anecdotes always pop up. A commenter notes how their personal experience contradicts the findings, bringing a bit of real life into the discussion. You, the reader, (being thoughtful and open-minded) add these anecdotes to your compilation of thoughts on the subject. You probably feel, at least subconsciously, like you have a more balanced, insightful view of the topic. Unfortunately, your view is not at all balanced, from a purely rational perspective. The anecdote carries a grossly exaggerated weight in your mind. If you are a conscientious reader, ignoring these anecdotes and downvoting them where necessary is a crucial part of maintaining a healthy, rational discussion.

Anecdotal evidence has been shown to have a greater influence on opinion than it logically deserves, most visibly when the anecdote conflicts with the reader’s opinion and when the reader is not highly analytical, even if the anecdotes are accompanying statistical evidence. Though the anecdotes may not totally sway you, they can easily leave you with the sense that the research findings aren’t as conclusive as they claim to be.

For example, if a close friend goes on and on about how the Ford he bought was a piece of crap, detailing how the transmission failed at 30k miles and the rear-view mirror fell off, you’ll be wary about buying a Ford in the future, even if Consumer Reports rates them highly. Your friend’s anecdote is a true story, certainly, but it’s bad evidence for several reasons: it’s subject to confirmation bias, the availability heuristic, and the data ultimately has a sample size of 1.

Contrarian anecdotes like these are particularly common in medical discussions, even in fairly rational communities like HN. I find this particularly insidious (though the commenters mean no harm), because it can ultimately sway readers from taking advantage of statistically backed evidence for or against medical cures. Most topics aren’t as serious as medicine, but the type of harm done is the same, only on a lesser scale.

In the absence of strong evidence, especially in new or uncommon areas, anecdotes may be the best thing you can get. But in the presence of statistical evidence, don’t tolerate contrarian anecdotes, and don’t make them yourself, knowing the exaggerated impact they can have. If you want to advance sound knowledge within the community, it might feel mean, but do your duty, and downvote those anecdotes.

EDIT: There’s also a good discussion on Hacker News