Test your Science Literacy Skills
Last week an interesting paper was published outlining the development of a test designed to evaluate science literacy. The basic idea of the test is to examine a number of different skills that are involved in evaluating scientific claims and facts.
I liked this idea so much I decided to put together an online version of the quiz that people could take and get instant feedback for. This could also be adapted for use in an educational environment as intended in the original paper.
To that end I need beta testers, people who would like to take the test and give me feed-back on how to improve it. A couple of things I need to mention at this point:
First, the test will ask for your email address in order to send a summary of how well you did on the test. Your email address will not be used for any other purpose and indeed I’ve set up the back-end so that the address will be deleted as soon as your email is sent – your private information will remain that way.
With this in mind – If you have feed-back about your performance, there is no way for me to tell which entry is yours unless you give me the time you submitted the form.
Secondly, I have used some custom html code in creating the form which doesn’t integrate seamlessly with the google drive document that runs most of the functionality. This means if you miss a required question (all of them are required FYI) than you will be re-directed to the original form of the test, this will look a bit different and will not contain the pictures (but will have urls you can use to get to them).
You can either continue with this version of the form or hit your browser’s Back button and fill out the questions you missed there.
Finally, If you have and comments or suggestions leave them here and I’ll see what I can do. Also I am actually fairly mediocre at html coding and such like so if there is anyone who wants to volunteer their services to upgrade the quiz I’m happy to share the load 
.
Ok, here’s the link to get you started, have fun.
[Edit: Here's a link to some initial analysis of the results]
Interesting. Have passed on to daughter doing GCSE exams at school in U.K. Suggested she shows to Science Teachers and see if something of interest for them to use. Managed 25 out of 28 which was quite pleased with for a non-scientist. Guess at least some of my skeptical reading has sunk in, along with a smattering of school learning many years ago. Would be great if it spread further. Has it been touted at JREF? Maybe send to them?
Yeah, I got 25/28 too.
.
Showing a science background isn’t necessarily beneficial either
Would be great if more teachers got to see this so thanks for that.
JREF huh? good suggestion. I don’t have much to do with those folks directly (though I think they’re great) so I’ll have to find an “in”. You you’ve already got one feel free to send details..
Thanks a lot.
Interesting idea and good to see some effort in this area. Some feedback regarding the test:
- My score was 24/28 and I am a working scientist
- IMO, the wording of some questions is confusing (e.g. Q4, Q7, Q14, Q21), perhaps due to questions phrased in the negative, e.g. “which of the following is *not* a strength of the trial” rather than “which of the following are weaknesses of the trial”.
- In Q14, none of the options seem to be strengths of the trial (2500 isn’t that large, there’s no indication that participants were chosen randomly, and participants were not randomly assigned to groups). However, “All of the above” was marked as incorrect in my test. It would be good to clarify this question.
- In Q15, it’s debatable whether graph C or D is better. No information is given about how stress was measured in the study. One would hope real studies are not so crude as to characterize a complex issue like stress as simply “high” or “low”. I think a good study should be designed to produce a graph like option C (this answer is marked as incorrect)
- The quiz is oriented strongly towards biological sciences. It would be nice to include questions from the physical sciences.
- The quiz would be quite challenging for members of the general public
Hope this feedback is useful.
RE ambiguous wording. Agreed, see today’s post (http://scepticon.wordpress.com/2013/01/23/science-literacy-test-initial-follow-up/)
I think diversifying the question topics would be a great idea. I suspect the bio focus is a consequence of the specialities of the initial authors, got any suggestions?
Spread the quiz to more of the public and maybe we’ll find out.
Great feedback, thanks.
Awesome quiz – What are you planning to do with the results? I would be curious to see a split of the results between different demographics, particularly people with a science vs non-science background, so maybe you could add some fields to collect that data also?
I’m thinking so, initially this was just supposed to be a de-bugging run and to see what people thought of the questions. But it’s taken off a bit so I am thinking of adding demographic data. Trouble is now that it’s getting new entries every couple of minutes it’s kind of hard to update
.
I’ll see if I can update an offline version an then quickly swap out bits all at once…
Fun quiz. Do you think that Q22 should stipulate that the outside researchers are neutral?
Good question. I think the question works as it is (ie even if the scientists weren’t neutral they might still be more reliable than the other sources) but it is something that is assumed by “outside” rather than explicitly stated for complete clarity.
So thanks for that, I was just mentioning on another site that I wanted insights exactly like this one.