In discussions on the findings of a piece of research, a
handful of contrarian anecdotes always pop up. A commenter notes
how their personal experience contradicts the findings,
bringing a bit of real life into the discussion. You, the reader,
(being thoughtful and open-minded) add these anecdotes to your
compilation of thoughts on the subject. You probably feel, at
least subconsciously, like you have a more balanced, insightful view
of the topic. Unfortunately, your view is not at all balanced,
from a purely rational perspective. The anecdote carries a grossly
exaggerated weight in your mind. If you are a conscientious reader,
ignoring these anecdotes and downvoting them where necessary is a
crucial part of maintaining a healthy, rational discussion.
Anecdotal evidence has been shown to have a greater influence on
opinion than it logically deserves, most visibly when the anecdote
conflicts with the reader’s opinion and when
the reader is not highly analytical, even if the anecdotes are
accompanying statistical evidence. Though the anecdotes may not totally
sway you, they can easily leave you with the sense that the research
findings aren’t as conclusive as they claim to be.
For example, if a close friend goes on and on about how the Ford he
bought was a piece of crap, detailing how the transmission failed at
30k miles and the rear-view mirror fell off, you’ll be wary about
buying a Ford in the future, even if Consumer Reports rates them
highly. Your friend’s anecdote is a true story, certainly, but it’s bad
evidence for several reasons: it’s subject to confirmation bias,
the availability heuristic, and the data ultimately has a sample size
Contrarian anecdotes like these are particularly common
in medical discussions, even in fairly rational communities like HN.
I find this particularly insidious (though the commenters mean no harm),
because it can ultimately sway readers from taking advantage of
statistically backed evidence for or against medical cures. Most topics
aren’t as serious as medicine, but the type of harm done is the same,
only on a lesser scale.
In the absence of strong evidence, especially in new or uncommon areas,
anecdotes may be the best thing you can get. But in the presence of
statistical evidence, don’t tolerate contrarian anecdotes, and don’t
make them yourself, knowing the exaggerated impact they can have.
If you want to advance sound knowledge within the community, it
might feel mean, but do your duty, and downvote those anecdotes.
EDIT: There’s also a good discussion on Hacker News