Posts Tagged ‘ Skeptical Inquiry ’

“I was a Skeptic, but…”

I wish people wouldn’t say this, it’s usually followed by some lame reason why we should trust their anecdotal experience over empirical data. Sure the word skeptic (or sceptic if you prefer) has a certain colloquial definition and to a large extent words are defined by the way they are used, I mean no-one uses the word “gay” to refer to being happy anymore.

Even so this usage is getting on my nerves. When I use the word “skeptic” to refer to myself I mean someone who evaluates the available evidence and comes to a reasonable conclusion. Implicit in my definition is also  an understanding of human foibles with regard to cognitive biases and a deep seated inability to view our own experiences impartially. Refer to my previous post for more in this vein.

On a whim I thought I’d look up what on-line dictionaries had to say about the word, I found some variation of the following to be popular:

1. One who instinctively or habitually doubts, questions, or disagrees with assertions or generally accepted conclusions.”

That doesn’t seem any better to me. So, what’s my problem?

Well for a start those alluded to in the title of this post are not applying skepticism they are merely doubtful. And when evaluating claims they are not using the methods of science they are using the unreliable guide that is personal experience. Thus, while their protestations of skepticism and subsequent conversion sound impressive, they are (to my ears) merely the hollow echo of true inquiry.

Harsh enough for you? well perhaps. I don’t expect all who use the word skeptic to apply to it the same definition that I do, but it still chafes.

The dictionary definition given above is also lacking in nuance, it appears more suited to define a contrarian than skeptic. What my favourite skeptical interviewer DJ Grothe refers to as “knee-jerk skepticism”. A skeptic isn’t someone who just says “no”, a skeptic is someone who asks “how do we know?”.

The reason my hypothetical skeptical convert gets on my wick so much is when answering the “how do we know?” question they assume that they can draw general conclusions from their informal experiment where n=1. This ties into the “don’t knock it ’til you’ve tried it” line of argument. NO. Trying it myself is not the way to determine the validity of a claim. This falls under the category of anecdote, and anecdotes are not good quality evidence. At best they should be the start of investigation – not the end.

When evaluating a claim we should look at two things in particular, yes we should determine the direct evidence for the claim i.e. is there evidence to show that it acts as claimed? but we should also attempt to see how the specific claim fits into the wider scientific ecosystem – the prior probability if you will.

Often in day to day claims this is of little practical importance and so it becomes overlooked when it is relevant. A new gadget or medication is often based on previous iterations of the same technology or medical practice and represents an incremental improvement or merely an additional option in the sphere or possibilities. However some claims are sufficiently far from mainstream understanding that we should take a step back and consider the likelihood that the claim is possible, irrespective of the evidence presented for the claim itself.

In the case of say, homeopathy or power balance bands our current understanding of the science should make us extremely wary of efficacy even before the specific claims are considered. To be clear here though, plausibility should be used as only part of the process, there are many things that work without us knowing how they work but the further outside of current knowledge something is the stronger the evidence we should require before we accept it. Certainly for many “alternative” therapies that strong evidence simply does not exist, as I presented for Amber teething beads there is no reason to think they should work from a physical or medical point of view so our standard of evidence should be higher than the earnest assurances of people in mothering forums, or even our own experience – as noted above.

But this is exactly the sort of pseudo-evidence that we are wired to find most convincing. Throughout most of our history the ability to evaluate randomised trials, statistics and p-values would not have aided our survival one whit. Therefore it’s not surprising that most of us are bad at it.*

Yes, it’s hard. Yes, it requires work, and yes you will probably get it wrong most of the time.** But it’s worth it. So give it a try – be skeptical, like you mean it.


* Arguably all of us, it requires practice and even the “experts” can get it wrong.

* I certainly do.

Enhanced by Zemanta

Five Signs You Might Be Wrong

Over the last few weeks several things have been happening both in my own life and in the wider public (Ken Ring) that have made me think about good rules of thumb to determine whether a claim is likely to be right or wrong. In particular when is it reasonable to perform a self examination and ask the question “Could I be wrong?”.

We have to make decisions with limited information every day, it’s not a good idea to be frozen with existential doubt over every little factoid wondering if the entire basis of knowledge is up to the task of determining Truth. I suspect society would grind to a halt if everyone suddenly started doing this.

But, it is a good idea to clear the cobwebs every once in a while and put ourselves back on a firm footing. What sort of things should we be on the lookout for in our mental spring cleaning? I don’t think I have the definitive answer but here are a few ideas that have presented themselves to me lately as a decent place to start:

5. Everyone knows that or “Common Wisdom”.

These are thing that you just absorb from the culture, you don’t know where you heard them but it’s so ingrained it just seems like common sense. Things like reading in low light being bad for your eyesight or only using ten percent of our brains. These permeate our popular consciousness like the air we breath.

Common myths get perpetuated in this way, you haven’t looked into it but you just know, that’s how the world works. This is a form of argument from popularity, reasonable most of the time but not a method of generating knowledge that is optimised for accuracy. On one level this is fairly harmless, usually these things don’t impact the major decisions in life and you aren’t especially invested in a particular conclusion. If you don’t really lose most of your body heat through your head, well it doesn’t matter, you were going to wear a hat anyway – it looks nice.

On the other hand this can be the most insidious method of creating misconceptions. Much of the time you aren’t even aware of them, if you were raised in Japan then you might just “know” that women are subservient to men. If that seems too drastic, perhaps you just have a general feeling that boys are better at maths and science than girls. These are things that we implicitly learn from our culture and can be difficult to dispel even if we are aware of the actual facts.

4. You learned about it from a Chain email

You’ve seen them. Emails that have been forwarded from one person to the next, each one thinking that someone earlier in the chain has probably checked it out and besides “what if” it’s true. Better send it on just in case.

I’ve seen several of these lately, see my report on Lemons and Cancer. Another has been making the rounds post earthquake ostensibly describing how to survive a a serious earthquake. This also has misleading and possibly dangerous information if the advice is followed. Basically I view everything transmitted in this manner as suspect until proven otherwise, the 21st century version of word of mouth seems to by-pass both verification and common sense because it is so easy to press the forward button and we appear to still afford the written word a default respect that it may not deserve (though you should respect my written words, ‘cos I say so).

What I find most frustrating about this phenomenon is that de-bunking these things is almost as easy as hitting that forward button. It usually takes me about 20 seconds – depending on how my internet connection is faring that day – to copy and paste some of the text into google or check on a site like

3. You’re on the edge or just beyond what we currently know scientifically.

Here is where things start taking a turn toward the dark side. Prior to this stage we could just look up the correct information and set ourselves straight. At this point though we need to start applying actual critical thinking and assess new information on it’s plausibility and merits. Sometimes this is just extrapolating from a recent breakthrough (invisibility cloaks anyone?), this tends to be easily recognised and dismissed.

We know that when working on the frontiers of knowledge many of our conclusions are false, or wildly simplistic. Forming opinions on the back of these initial forays into the unknown is therefore fraught with peril for the unwary, or even the wary for that matter. This is where some quacks can move in, taking preliminarily positive results for some treatment or technology based on initial tests and making claims that are not backed by sound data. We could put “black market” stem cell therapies or cancer cures into this category.

Alternatively there are some who take plausible trends in scientific and technological progress and predict specific technologies will be developed in specific time frames (looking in your direction Kurzweil). Others may claim that because our knowledge is underdeveloped, either in general or in a particular discipline, that their pet theory should be given a pass.

These ideas may or may not be correct but we can make reasonable determinations as to likelihood based on current scientific knowledge, science can bring up counter-intuitive facts about our universe but in general we tend to see incremental advancement on existing knowledge. Self replicating machines? perhaps, perpetual motion machines? no, not really.

In this way this category can tend to bleed into my next warning sign…

2. Your point of view goes against/disagrees with a large proportion of scientists/medical professionals.

This is the point where you are starting to cross over to crankery. This is where warning sirens should start going off in your head when you are confronted with “alternative” theories. Sometimes though it is not obvious straight away, only once we have accepted and become invested in a theory do we get confronted with disconfirming evidence. We should not be afraid to let go of ideas when they are shown to be incorrect.

Examples of this might be that vaccines cause autism, or that fluoridation is harmful. The key here is that it is not prima facie impossible that these things are true, but the preponderance of data has shown that they are very unlikely to be true. At this point it is incumbent upon a reasonable person to change their mind.

Once you decide that you are either better informed than the scientific or medical community or that there exists a conspiracy to keep these things from the public you start to cross the line between concerned citizen to outright crank. It’s possible that you really are a genius, that the scientists really are wrong, that you really are right, that you really can compare yourself favourably to Galileo, but really – it’s not likely. Luckily, you can come back from the edge – it takes courage though. One thing to keep in mind is that you should be committed to the enquiry, not the conclusion. This way you should be able to  follow where the evidence leads and change you opinion accordingly.

1. Large chunks of science have to be wrong for you to be right

Congratulations, you have graduated to fully fledged crank. You are immune to evidence and reason and live in your own self-contained universe of nonsense, insulated from reality by your enormous  self-righteousness ego. Examples of this extreme form of scientific inaccuracy are Creationism, Neal Adams’ “Growing Earth” theory, and Homeopathy. If you have made it this far then chances are you are so committed to your ideas that no amount reasoned argumentation will sway you, you may have a tinfoil hat somewhere on your person right now.

But, and let me make this clear, you are NOT an idiot*. You have just invested so much in a particular point of view that changing you mind now would tantamount to repudiating a large portion or your life. That’s painful and not a course of action anyone wants to undertake.

At this point it becomes farcical to even suggest that you might be right and the accumulated knowledge of the last 200-400 years is wrong. Recall how I said earlier that we make incremental advances based on existing knowledge? Well all of our past discoveries have inexorably been leading to the fact can';t infinitely dilute a substance and still expect an effect…the earth really is 4, 500,000,000 years old…energy really can’t be created from nothing… and on and on.

All that said, again,  it still is possible for you to change your mind and come back to reality. A number of people have done it, but the chances do become lower. Good luck.


The above should be taken as a light-hearted look at our foibles, no real offence is intended. – He said, trying futility to ward off trolls.

* Ok, I can’t back that up. Some of you are bound to be idiots… just sayin’…statistically… you know.

Enhanced by Zemanta

A Skeptic’s Perspective – Repost

After Friday’s unabashed rant on things magical and absurd I felt it would be good to post a more level headed and rational piece. This post from a couple of years ago caught my eye at the right time and given the popularity of reposting I decided it was worth another outing:

A less formal post today, unstructured playtime if you like. This week I had not had a chance to write my usual contribution to my work newsletter. Instead I decided to search for a skeptical cartoon to use as a place holder until next week. I found two things, first that there aren’t very many good skeptically themed ‘toons around and second those I could find just seemed to portray skepticism as simple doubting. While this is a good place to start it certainly isn’t all there is to the skeptical outlook. While philosophical skepticism is concerned with whether there is any such thing as objective reality the modern Skeptical movement takes this for granted and seeks to supply a basis for rational inquiry and thought.

A skeptic attempts to examine claims objectively, trying to limit the effect of bias on their conclusion, both that of the person making the claim and their own. This is obviously not easy, we all view the world through a collection of filters that encompass all of our preconceptions, hopes, wants and needs as well as, for most of us, a desire to be part of a larger community. This can seriously affect how we interpret the events around us and hamper our ability to make sound judgments concerning the validity of claims that we make and that are presented to us a factual. As I have previously discussed personal experience as related through anecdote is extremely powerful in affecting the way people perceive the world. An emotion filled story about someone’s plight with an illness and subsequent miraculous recovery is often enough to convince most people of the efficacy of a treatment.

For a skeptic however, certain minimum standards of evidence must be met before we can evaluate the plausibility of a claim. The more unusual or outrageous the claim the higher that minimum standard becomes. This is because prior plausibility must be considered as part of the evaluation process. We all do this in our lives, when we consider how likely a friend is to keep that promise to arrive on time we think back to previous occasions when they were early or late and decide how likely it is that they will make it this time. We consider the prior probability to their claim of punctuality. A friend who is habitually late will score lower that one who has proven time and again that they can read a watch. In the same way a skeptic will consider the evidence that has come before when looking at the claims of Homeopathy, Chiropractic, Therapeutic Touch, psychics, mediums, cryptozoologists, etc.

So, if you meet a skeptic remember that they are not simply trying to ridicule paranormal experience but are attempting to apply reason to the world around them and are hoping that they are providing an example for others to do so as well. At least that’s the way I see it. Happy critical thinking.

Enhanced by Zemanta

Get every new post delivered to your Inbox.

Join 73 other followers