Posts Tagged ‘ Scientific method ’

The Scientific Method On TV


Last week Michael Edmonds asked “Which TV Show Best Demonstrates the Scientific Method?“
Various shows where suggested such as CSI and NCIS. Bones made an appearance in the comments as one show that exhibits a fair amount of pseudoscience along with it’s “real” science. House was praised for it’s attention to hypothesis generation and testing. I generally agree with that but found the fact that House always came up with the correct answer alone and via an epiphany type insight a bit unsatisfying.

In any case most shows do poorly at portraying science, this comes inevitably out of the fact that the show is there to tell a story. The science may or may not help with that but in the end it is merely set dressing for the real aim. I’m fine with that – I watch a lot of tv and aside for the odd grumble along the lines of “It would take longer than that!” or “You wouldn’t do it that way!” I’m happy enough to suspend my criticism and enjoy the ride.

But the question remains – which show does it best? Over the weekend an answer came to me that might be cheating a bit, but I think the best I’ve seen lately is one called “Guess with Jess“*.
Guess with Jess

If it sounds like a kids show, you’re right – it is.

The basic set up is like this: A cat decides on a question for the day and then sets about trying to answer it. Pretty simple.

I think it does pretty well showing the methods of science:
A question is generated via an observation of the world, a “literature” search is conducted to find what is already known on the topic (via asking the other animals), a hypothesis is generated that fits the question and what is known, the hypothesis is tested and the results observed to see if it answers the question. Often the first attempt is incorrect, so the question is refined, another “literature” search is conducted, another hypothesis generated and another round of testing conducted.

At the end an answer is arrived at which satisfies our feline protagonist and everyone is happy at having learned something new. Possibly Jess goes off to write a grant proposal – I’m not sure.

So that’s my answer. Adult fiction is too focused on telling a compelling story with relate-able characters in a limited time frame to make more than a passing effort at getting the science right. But that’s ok, we can rely on a young cat named Jess to pick up the slack.

———————————————————————————————

* If a black and white cat named Jess sounds familiar, you might remember that this describes the cat of Postman Pat. According to wikipedia they are one in the same, I’m not sure if Jess has been put out to pasture or if this depicts Jess’s life before settling down with Pat.

Enhanced by Zemanta
2013
04/29
CATEGORY
Sciblogs
Science
Leave a comment

The Role of Experiments in Science


In an attempt to widen my horizons I have been listening to the Philosophy Now podcast out of the UK. A recent episode concerned philosophy of science (ep 29) and the host asked an interesting question: “Why do we have to have experiments when sometimes we can get away with observations?”.

The point he was trying to make was that when we think about science in an abstract way we also have the idea that experiments are at the core of the scientific method. This clashed somewhat with the person the host was talking with at the time who was emphasising observation as the way we confirm hypotheses in science.

The implication here is that we have two things, observations on the one hand and experiments on the other.

This appears to be a fairly common view, I have seen arguments accusing cosmology (specifically the big bang theory) of not being science because you can’t perform an experiment to create a new universe. Similar arguments have been made for evolution.

I don’t know how widely held this view is in the general population (as opposed to those who are set against certain findings of science) but the question of the podcast host implies that it’s wide enough.

The problem with this view however is that there really aren’t two things here that are different in kind. Rather, one is a sub-set of the other; experiments are a special kind of observation.

The whole point of an experiment is to interrogate nature in a specific kind of way. While we can passively observe an event and gain valuable information (say, watching the development of an embryo) we can also create an experiment that constrains the conditions in a particular way in order for us to draw more conclusive conclusions about the situation of interest (perhaps we knock out a gene and watch that embryo follow a different developmental path).

By using experiments we aren’t doing anything fundamentally different, we are still observing what nature has to tell us about the world we inhabit, but we are trying to set up conditions that are meant to clarify what nature is saying. In this view experiments are nature’s interpreter.*

Experiments also allow us to get access to things that we might not normally be able to see. For example high energy physics requires elaborate experiments in order to allow us to in some way visualise particles that are mind bogglingly small. We aren’t creating the physics we observe we are simply delving into realms that would normally be hidden from us.

This was brought home to me a few years back when the attempts to listen for extraterrestrial signals by SETI were referred to as experiments. In this case we aren’t setting up the conditions by which we control whether an ET sends us a signal, we are determining the conditions by which we would receive such a signal. At it’s heart this activity is an observation, no different in it’s intent from viewing a microbe under a microscope.

So it is that the ability to do or not do a experiment does not determine science from non-science (termed the demarcation problem and certainly not definitively settled). Experiments may have come to be thought of as the defining feature of science but they are really just a special case of something we all do every day – observe the world around us.

—————————

*If that’s too narrow for you, how about experiments as nature’s speech therapist?

Enhanced by Zemanta
2012
03/20
CATEGORY
Sciblogs
Science
skepticism
Leave a comment

“I was a Skeptic, but…”


I wish people wouldn’t say this, it’s usually followed by some lame reason why we should trust their anecdotal experience over empirical data. Sure the word skeptic (or sceptic if you prefer) has a certain colloquial definition and to a large extent words are defined by the way they are used, I mean no-one uses the word “gay” to refer to being happy anymore.

Even so this usage is getting on my nerves. When I use the word “skeptic” to refer to myself I mean someone who evaluates the available evidence and comes to a reasonable conclusion. Implicit in my definition is also  an understanding of human foibles with regard to cognitive biases and a deep seated inability to view our own experiences impartially. Refer to my previous post for more in this vein.

On a whim I thought I’d look up what on-line dictionaries had to say about the word, I found some variation of the following to be popular:

“1. One who instinctively or habitually doubts, questions, or disagrees with assertions or generally accepted conclusions.”

That doesn’t seem any better to me. So, what’s my problem?

Well for a start those alluded to in the title of this post are not applying skepticism they are merely doubtful. And when evaluating claims they are not using the methods of science they are using the unreliable guide that is personal experience. Thus, while their protestations of skepticism and subsequent conversion sound impressive, they are (to my ears) merely the hollow echo of true inquiry.

Harsh enough for you? well perhaps. I don’t expect all who use the word skeptic to apply to it the same definition that I do, but it still chafes.

The dictionary definition given above is also lacking in nuance, it appears more suited to define a contrarian than skeptic. What my favourite skeptical interviewer DJ Grothe refers to as “knee-jerk skepticism”. A skeptic isn’t someone who just says “no”, a skeptic is someone who asks “how do we know?”.

The reason my hypothetical skeptical convert gets on my wick so much is when answering the “how do we know?” question they assume that they can draw general conclusions from their informal experiment where n=1. This ties into the “don’t knock it ’til you’ve tried it” line of argument. NO. Trying it myself is not the way to determine the validity of a claim. This falls under the category of anecdote, and anecdotes are not good quality evidence. At best they should be the start of investigation – not the end.

When evaluating a claim we should look at two things in particular, yes we should determine the direct evidence for the claim i.e. is there evidence to show that it acts as claimed? but we should also attempt to see how the specific claim fits into the wider scientific ecosystem – the prior probability if you will.

Often in day to day claims this is of little practical importance and so it becomes overlooked when it is relevant. A new gadget or medication is often based on previous iterations of the same technology or medical practice and represents an incremental improvement or merely an additional option in the sphere or possibilities. However some claims are sufficiently far from mainstream understanding that we should take a step back and consider the likelihood that the claim is possible, irrespective of the evidence presented for the claim itself.

In the case of say, homeopathy or power balance bands our current understanding of the science should make us extremely wary of efficacy even before the specific claims are considered. To be clear here though, plausibility should be used as only part of the process, there are many things that work without us knowing how they work but the further outside of current knowledge something is the stronger the evidence we should require before we accept it. Certainly for many “alternative” therapies that strong evidence simply does not exist, as I presented for Amber teething beads there is no reason to think they should work from a physical or medical point of view so our standard of evidence should be higher than the earnest assurances of people in mothering forums, or even our own experience – as noted above.

But this is exactly the sort of pseudo-evidence that we are wired to find most convincing. Throughout most of our history the ability to evaluate randomised trials, statistics and p-values would not have aided our survival one whit. Therefore it’s not surprising that most of us are bad at it.*

Yes, it’s hard. Yes, it requires work, and yes you will probably get it wrong most of the time.** But it’s worth it. So give it a try – be skeptical, like you mean it.

————————————–

* Arguably all of us, it requires practice and even the “experts” can get it wrong.

* I certainly do.

Related articles
Enhanced by Zemanta
2011
04/24
CATEGORY
Sciblogs
skepticism
1 comment

Five Signs You Might Be Wrong


Over the last few weeks several things have been happening both in my own life and in the wider public (Ken Ring) that have made me think about good rules of thumb to determine whether a claim is likely to be right or wrong. In particular when is it reasonable to perform a self examination and ask the question “Could I be wrong?”.

We have to make decisions with limited information every day, it’s not a good idea to be frozen with existential doubt over every little factoid wondering if the entire basis of knowledge is up to the task of determining Truth. I suspect society would grind to a halt if everyone suddenly started doing this.

But, it is a good idea to clear the cobwebs every once in a while and put ourselves back on a firm footing. What sort of things should we be on the lookout for in our mental spring cleaning? I don’t think I have the definitive answer but here are a few ideas that have presented themselves to me lately as a decent place to start:

5. Everyone knows that or “Common Wisdom”.

These are thing that you just absorb from the culture, you don’t know where you heard them but it’s so ingrained it just seems like common sense. Things like reading in low light being bad for your eyesight or only using ten percent of our brains. These permeate our popular consciousness like the air we breath.

Common myths get perpetuated in this way, you haven’t looked into it but you just know, that’s how the world works. This is a form of argument from popularity, reasonable most of the time but not a method of generating knowledge that is optimised for accuracy. On one level this is fairly harmless, usually these things don’t impact the major decisions in life and you aren’t especially invested in a particular conclusion. If you don’t really lose most of your body heat through your head, well it doesn’t matter, you were going to wear a hat anyway – it looks nice.

On the other hand this can be the most insidious method of creating misconceptions. Much of the time you aren’t even aware of them, if you were raised in Japan then you might just “know” that women are subservient to men. If that seems too drastic, perhaps you just have a general feeling that boys are better at maths and science than girls. These are things that we implicitly learn from our culture and can be difficult to dispel even if we are aware of the actual facts.

4. You learned about it from a Chain email

You’ve seen them. Emails that have been forwarded from one person to the next, each one thinking that someone earlier in the chain has probably checked it out and besides “what if” it’s true. Better send it on just in case.

I’ve seen several of these lately, see my report on Lemons and Cancer. Another has been making the rounds post earthquake ostensibly describing how to survive a a serious earthquake. This also has misleading and possibly dangerous information if the advice is followed. Basically I view everything transmitted in this manner as suspect until proven otherwise, the 21st century version of word of mouth seems to by-pass both verification and common sense because it is so easy to press the forward button and we appear to still afford the written word a default respect that it may not deserve (though you should respect my written words, ‘cos I say so).

What I find most frustrating about this phenomenon is that de-bunking these things is almost as easy as hitting that forward button. It usually takes me about 20 seconds – depending on how my internet connection is faring that day – to copy and paste some of the text into google or check on a site like Snopes.com.

3. You’re on the edge or just beyond what we currently know scientifically.

Here is where things start taking a turn toward the dark side. Prior to this stage we could just look up the correct information and set ourselves straight. At this point though we need to start applying actual critical thinking and assess new information on it’s plausibility and merits. Sometimes this is just extrapolating from a recent breakthrough (invisibility cloaks anyone?), this tends to be easily recognised and dismissed.

We know that when working on the frontiers of knowledge many of our conclusions are false, or wildly simplistic. Forming opinions on the back of these initial forays into the unknown is therefore fraught with peril for the unwary, or even the wary for that matter. This is where some quacks can move in, taking preliminarily positive results for some treatment or technology based on initial tests and making claims that are not backed by sound data. We could put “black market” stem cell therapies or cancer cures into this category.

Alternatively there are some who take plausible trends in scientific and technological progress and predict specific technologies will be developed in specific time frames (looking in your direction Kurzweil). Others may claim that because our knowledge is underdeveloped, either in general or in a particular discipline, that their pet theory should be given a pass.

These ideas may or may not be correct but we can make reasonable determinations as to likelihood based on current scientific knowledge, science can bring up counter-intuitive facts about our universe but in general we tend to see incremental advancement on existing knowledge. Self replicating machines? perhaps, perpetual motion machines? no, not really.

In this way this category can tend to bleed into my next warning sign…

2. Your point of view goes against/disagrees with a large proportion of scientists/medical professionals.

This is the point where you are starting to cross over to crankery. This is where warning sirens should start going off in your head when you are confronted with “alternative” theories. Sometimes though it is not obvious straight away, only once we have accepted and become invested in a theory do we get confronted with disconfirming evidence. We should not be afraid to let go of ideas when they are shown to be incorrect.

Examples of this might be that vaccines cause autism, or that fluoridation is harmful. The key here is that it is not prima facie impossible that these things are true, but the preponderance of data has shown that they are very unlikely to be true. At this point it is incumbent upon a reasonable person to change their mind.

Once you decide that you are either better informed than the scientific or medical community or that there exists a conspiracy to keep these things from the public you start to cross the line between concerned citizen to outright crank. It’s possible that you really are a genius, that the scientists really are wrong, that you really are right, that you really can compare yourself favourably to Galileo, but really – it’s not likely. Luckily, you can come back from the edge – it takes courage though. One thing to keep in mind is that you should be committed to the enquiry, not the conclusion. This way you should be able to  follow where the evidence leads and change you opinion accordingly.

1. Large chunks of science have to be wrong for you to be right

Congratulations, you have graduated to fully fledged crank. You are immune to evidence and reason and live in your own self-contained universe of nonsense, insulated from reality by your enormous  self-righteousness ego. Examples of this extreme form of scientific inaccuracy are Creationism, Neal Adams’ “Growing Earth” theory, and Homeopathy. If you have made it this far then chances are you are so committed to your ideas that no amount reasoned argumentation will sway you, you may have a tinfoil hat somewhere on your person right now.

But, and let me make this clear, you are NOT an idiot*. You have just invested so much in a particular point of view that changing you mind now would tantamount to repudiating a large portion or your life. That’s painful and not a course of action anyone wants to undertake.

At this point it becomes farcical to even suggest that you might be right and the accumulated knowledge of the last 200-400 years is wrong. Recall how I said earlier that we make incremental advances based on existing knowledge? Well all of our past discoveries have inexorably been leading to the fact that..you can’;t infinitely dilute a substance and still expect an effect…the earth really is 4, 500,000,000 years old…energy really can’t be created from nothing… and on and on.

All that said, again,  it still is possible for you to change your mind and come back to reality. A number of people have done it, but the chances do become lower. Good luck.

——————————————-

The above should be taken as a light-hearted look at our foibles, no real offence is intended. – He said, trying futility to ward off trolls.

* Ok, I can’t back that up. Some of you are bound to be idiots… just sayin’…statistically… you know.

Related articles
Enhanced by Zemanta
2011
03/25
CATEGORY
Psychological
Sciblogs
Science
skepticism
2 comments

Dig-in Or Adapt: The Effect of Political Views on Changing One’s Mind


Are you the sort of person who will change their position on a subject conditional on further information gained? I personally think this is the behaviour we should all be striving for. If we do not we end up withdrawing further and further from reality, insulating ourselves from rational argument and retreating into our own self contained worlds of comforting fantasy. Not unlike a Pauly Shore fan.*

Seriously though, when confronted with information that contradicts previous knowledge there are a number of different reactions possible. One is to reject the previous knowledge in favour of the new information, there are good reasons why we shouldn’t do this uncritically. If we simply accepted every new thing we were exposed to we would become like motes in the wind, changing direction constantly. Long term decisions and actions would become impossible and our lives would be subject to the merest whim. A poor strategy.

A different approach is to consider the new information in context with previous knowledge, determine which is more likely to be correct and then determine our actions accordingly. This is the most favourable action in my view but may still result in a distressingly high chance of having to change your mind on a regular basis. Again, possibly making it difficult to forge and maintain long term alliances and life strategies. A variation on this theme might be strategically best.

Another option is to summarily reject any new fact or opinion that is counter to your own, while this may make life simpler it is also fraught with difficulties in that it can make us too rigid and resistant to change. This is a recipe for ultimate downfall. A variation on this is to not only discount opposing views but to actually reinforce your commitment to prior beliefs.

This last variation was investigated in the paper “When Corrections Fail: The Persistence of Political Misperceptions“. The primary experiment showing the effect involved subjecting 130 participants to fake news stories about the US invasion of Iraq. The news stories contained an  actual quote from then president Bush regarding the possible existence of WMDs and the risk that this capability would be passed to terrorist groups.

The stories were then split into two conditions, they either then included information from the Duelfer Report that stated there were no WMD stockpiles nor an active program to produce such directly prior to the US invasion or a version that omitted this information.

Subjects were then asked to rated how much they agreed (On a 5 point scale; “Strongly agree” to “Strongly Disagree”) with the following statement:

“Immediately before the U.S. invasion, Iraq had an active weapons of mass destruction program, the ability to produce these weapons, and large stockpiles of WMD, but Saddam Hussein was able to hide or destroy these weapons right before U.S. forces arrived.”

When the responses of the subjects who received the corrective information were plotted with regard to their political stance (Liberal or conservative) it was found that conservative subjects were more likely to agree with the statement, with more conservative subjects agreeing more strongly.

It would seem from this that new information which not only contradicts previous knowledge but clashes with deeply held ideological convictions will have what the researches termed a “Backfire” effect. In this case the information apparently strengthened the previously held belief.

Now the experimental conditions apparently favoured an especially strong effect on politically conservative individuals but I don’t think that those on the the opposite end of the political spectrum are immune to this phenomenon. A further experiment aimed at this group did not show a clear “Backfire” effect but it did show that increasingly liberal subjects were less likely to be affected by the corrective information, in this case regarding the incorrect notion of a stem cell ban in the US.

In conclusion I think that, as always, we should take care that we examine all of our beliefs critically and not only evaluate new information based on whether it conforms to those beliefs but also on how accurately it conforms to reality.

*I don’t really mean that.

Thanks to the Badscience blog for the topic. Read Ben Goldacre’s post here.

Nyhan, B., & Reifler, J. (2010). When Corrections Fail: The Persistence of Political Misperceptions Political Behavior, 32 (2), 303-330 DOI: 10.1007/s11109-010-9112-2

Related articles by Zemanta
Reblog this post [with Zemanta]
2010
05/07
CATEGORY
Psychological
Sciblogs
Science
skepticism
5 comments

What is the Harm of Alternative Medicine?


Yesterday fellow Sciblogger Grant posted about homeopathic medications in pharmacies and questioned the legitimacy of reputable organisations selling such patent snake oil. The comments to this entry reveal one of the most frustrating aspects of speaking out against unscientific medicine and can be summarised thusly: “I’m far too sophisticated to be taken in by this stuff myself but other people seem to like it and if it doesn’t work then what’s the harm?”.

This attitude is ever present and comes from a reasonable starting point i.e. everyone is entitled to their own opinion and it’s not my job to save them from themselves. I can totally get behind that, usually. When it comes to ineffective medications of the alternative variety however this impulse though understandable is misguided and I’d like to put down a few reasons why I think so, some are speculative but I think the possibility of harm is great enough that they deserve to be considered.

For a start there may well be direct harm caused by using alternative remedies. As there is little to no regulation of these medications then no proof of safety or efficacy is required for sale. Witness the Zicam debacle last year regarding a “homeopathic” cold medication.

Further more the possibility for indirect harm (as multiply alluded to by Grant) may be significant. In case your imagination is not up to the task I will outline a few ways this may be the case. For instance the underlying principles of something like homeopathy are no only unscientific they are in direct contradiction of the last 200 years of scientific understanding. If they are used as the basis of reasoning about health then the results can be more dire than someone getting a bad nights sleep (in the case of the homeopathic sleep aid Grant used as an example).

Use of these therapies for minor ailments by the “worried and wonky well” may increase the possibility they they will be used for more serious health issues where the results could be deadly.

Look no further than the position statement of the WHO regarding the use of homeopathy in the treatment of Malaria and AIDs (among other things). The consequences of such thinking could be incalculable in terms of human suffering and spread of disease. But what’s the harm, right?

Additionally it is one thing for adults to make an informed choice for themselves based on available evidence filter through their particular world view but what about when this choice id forced on their children? The recent case of parents being found guilty of manslaughter over giving homeopathic remedies to their sick daughter is a terrible reminder that sometimes it is innocent children that pay the price for people’s gullibility. But, you know, what’s the harm?

When ostensibly professional medical providers such as pharmacists sell demonstrably irrational treatments they lend credibility to them that the average person uses to base decisions on. I mean the wouldn’t sell it if it didn’t work, right?

So while I understand the commitment to individual autonomy and freedom of choice that leads to the “What’s the Harm?” question, I fail to see how this means that fraudulent therapies must be let off the hook simply because there is a demand for them.

This has been a more vitriolic post than I normally write but what’s the point of a blog if you can’t vent once in a while?

Reblog this post [with Zemanta]
2010
01/26
CATEGORY
Alternative medicine
Medicine
Questionable Techniques
Sciblogs
Science
skepticism
12 comments
Sciblogs.co.nz
Atmospheric CO2 data

Rhyming Prose

Vital impulse synaptically flow through neurons, til they go. Lost without a trace not to some other place. The end may be final but our legacy is ample to ensure our continued existence though our life's a single instance.
twitter / Scepticon

Subscribe to email alerts!

Want an Email when new articles are posted? Click Here: Subscribe to Scepticon by Email
View Darcy Cowan's profile on LinkedIn
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 New Zealand License.
The Out Campaign: Scarlet Letter of Atheism
Follow

Get every new post delivered to your Inbox.

Join 55 other followers