Monday, July 21, 2014

The problem with anecdotes

Years ago while weed-whacking, I mowed up a nest of yellow jackets. They were swarming all around me, but for some reason they didn't attack me, and I didn't get even a single sting. (I wrote about this on another post years ago to make a separate point.) It's a pretty incredible story, and it's even better to hear me tell it in person.

Unfortunately, I have absolutely no evidence that this happened.

Wait...what...how can that be? I lived through it! I was there! I saw the damned things! How can I say that there is no evidence?

Because that's all I have. Now, had somebody been filming it, then maybe I could at least provide you with something that you could verify for yourself. Or if I was willing to go and try it with other yellow jacket nests (I'm not, by the way) and demonstrate my ability to wreak havoc on them sans stings, then you'd have something to evaluate.

But all I have is my story. It's a cool story, but it's not evidence. I'm not saying that you shouldn't believe me, but considering that it's hardly an everyday sort of occurrence, I don't suppose that I'd blame somebody if they didn't.

All I have is an anecdote.

The problem with anecdotes is that people will often submit them AS evidence for whatever phenomenon they're trying to advance as being fact. I was watching a video yesterday between a "psychic" and a professional debunker (I included it below) and the debunker pointed out that there was no evidence. The "psychic" kept saying that there IS evidence, and she cited all of her personal stories (in rather vague terms) as "evidence". What she didn't understand is that that isn't evidence at all because nobody else has any way of confirming what happened.

As a skeptic, I've been told that I "don't accept evidence" for the supernatural when people insist that they've had all kinds of personal experiences from miracles to psychic phenomena to encounters with ghosts and demons. When I try to explain that "anecdotes aren't evidence", I don't seem to get anywhere. I also don't get anywhere when I point to an example of an anecdote for something that they don't believe (My favorite? Sammy Hagar being kidnapped by aliens) and ask them if they count that as "evidence" or not.

Because if anecdotes are evidence then ALL anecdotes must be evidence, not just the ones that confirm what you already believe.

Honestly, I do understand though why people get really insistent when you doubt their personal stories. They take it as a personal attack, as though you're calling them liars. Nobody likes that, but calling somebody a liar and pointing out that they have no evidence for something is not the same thing. Back to my story with the yellow jackets, I suppose I would get a bit annoyed if my close friends called me a liar when told them the story. However, if they point out that I don't have evidence, then why should I get mad? They'd be right.

Try not to misunderstand me here. Just because an anecdote isn't evidence that doesn't mean that it didn't happen, and ultimately it's up to us as to what we believe. For me, I try to have the evidence determine how strong my belief is. My yellow jacket story is unlikely, but it doesn't necessarily break any of the known laws of the universe, and there are various unknown factors that could have contributed to what happened. If I told you that aliens probed my mind though, I would hope that your skepticism would increase as if that story was true, it would certainly change everything we know and understand about the universe in which we live.

Lately, I've tried a new tactic when confronted with people who give anecdotes as evidence. Sometimes they'll just keep giving me more and more, as if ten anecdotes suddenly equal evidence. I have one online friend who apparently lives in the equivalent of Smallville, but instead of visits from various superheroes from the DC Universe, there are miracle healings left and right. I think that he was starting to get frustrated as I was questioning the veracity of these statements.

The strategy that finally seemed to work was when I said that I didn't accept those stories as evidence because I had no way of confirming them one way or the other. All I had was his story and no way to check on it. Evidence is something that can be verified by anybody, and until he gave me something that I could actually check on, I was going to have to remain skeptical.

That seemed to work a bit better. If you're a skeptic like me, and you find that saying "anecdotes aren't evidence" gets you nowhere, try explaining it in those terms. At least then the other person might understand why their story isn't convincing you of anything.

8 comments:

Jeff Grigg said...

I don't agree with the assertion that antidotes are not evidence. I think that they are, as supported by the fact that personal stories about what one has observed are accepted in courts as evidence.

But I do assert that personal stories, antidotes, are very weak evidence. They are weaker than independently verifiable evidence. They are weaker when related by another person. (...which is one reason why courts don't accept "hearsay." Another is that hearsay can't be cross-examined.)

Antidotal evidence is not sufficient to override more plausible explanations.

I (provisionally) accept your wasp story because it's plausible and I have weak reasons to believe that you would be honest about such things. I'd be more inclined to believe it if you had the chopped-up wasp nest, so that I could verify that its damage was consistent with that of a "weed whacker." Pictures might suffice. Dated pictures would be good.

If someone else said they saw the event, I'd be somewhat more inclined to believe.

But I could never have much confidence in your story based only on your word.

Lance Johnson said...

"I think that they are, as supported by the fact that personal stories about what one has observed are accepted in courts as evidence."

But don't they need corroborating evidence to go along with them?

Jeff Grigg said...

No, you don't need corroborating evidence. If Abe died from stabbing, and six people said that they saw you do it, and there was no other evidence supporting or contradicting your claim that you did not kill Abe... how do you think you'd fare?

(The reliability of the witnesses would probably be considered. And motive -- yours and theirs.)

Lance Johnson said...

What if there wasn't a dead body? Wouldn't that be needed?

And the stories would have to match up as well, wouldn't they?

Fidem Turbāre said...

For the courts, the requirement "beyond a reasonable doubt" is key, and the absence of a dead body can be an important factor.

(Unfortunately juries often aren't as keenly trained in skepticism as judges are supposed to be, so the risk of injustice is greater due to democratizing judgement, but but that's beyond the scope of this article.)

Fidem Turbāre, the non-existent atheist goddess said...

For the courts, the requirement "beyond a reasonable doubt" is key, and the absence of a dead body can be an important factor.

(Unfortunately juries often aren't as keenly trained in skepticism as judges are supposed to be, so the risk of injustice is greater due to democratizing judgement, but but that's beyond the scope of this article.)

Jeff Grigg said...

I had assumed a dead body. And the "expert" (the coroner) says that the body has knife wounds, and that the knife wounds were the cause of death. The jurors can't see the body. They can't independently verify that Abe died from stabbing. So, in a sense, it's just another story that someone is telling.

Jeff Grigg said...

In a sense, all science is story telling: The scientists tell you what experiments they ran, and what results they got. And it makes sense for you to believe them -- without verifying or rerunning all the experiments yourself.

That other scientists say that they conducted similar experiments with similar results does legitimately give you good reason to believe the first story.