Wednesday, March 02, 2011

Multiple Choice Quizzes are (a) Ditzy; (b) Irrational (c) (etc.)

Somehow I'm just now catching up with Slate piece on the pathetic flaws in the multiple choice quiz she had to pass in order to become a citizen.  I feel her pain: I served on a committee for a while years ago where we wrote multiple choice questions for the California "baby bar."  It was actually kind of fun: four of us would show up with four drafts and rip each other's work to shreds, a man's manly idea of a good time.  The trick was, of course to sandblast away every hint of ambiguity or potential for misunderstanding before the questions fell into the hands of the young, the litigious and the aspirational.

As I say I enjoyed it but one thing I learned for sure:  at least in the social sciences, it is impossible to write a good multiple choice question.  After agonizing effort, you maybe able to write a pretty-much-unambiguous question but it will be so stylized that it won't teach anybody about anything.  I suppose it might be different elsewhere: if I am asked to select the right value for "pi," I suppose I will know enough to check 3.13159... .  But law, never, and I am  sure the same goes for citizenship.

But then again, that's not the point.  Recall that success in education is learning how to psych the examiner.  So the student is looking, not for the correct answer, but the one the examiner most likely thought was correct.  This is the skill being tested by Ms. Linzer's citizenship test.  If she doubts it, she should read  Chapter 2 of Dixit and Nalebuff's Art of Strategy where they tell you to answer a multiple choice question correctly without even knowing the question  (the answer is "16π."). 

4 comments:

Allan Connery said...

You gave the wrong value of pi. It's 3.14159...

This confronts me with a multiple-choice question; were you:

a) Making a joke?
b) Testing to see if anyone is paying attention?
c) Forgetting the actual value of pi?

Actually, alternatives a and b could simultaneously be true, which:

a) Illustrates the point you were making
b) Leads us toward madness

But again, both alternatives could be true at the same time. Forget I spoke.

Buce said...

The correct answer is "d"--typo. But I think I'll leave it as is and see what comes of it.

Ebenezer Scrooge said...

How I wish I could recollect pi
Eureka! Cried the great inventor
Christmas pudding, Christmas pie
Is the problem's very center

A good mnemonic for pi to 20 decimal places. Ignore punctuation.

Buce said...

My friend David offers another:

How I need a drink, alcoholic of course after the heavy lectures involving quantum mechanics...