Thursday, April 25, 2013

The Underappreciated Skill of Writing Exams


My son is in middle school. His English teacher had assigned the class an “Interactive Notebook” where they are to record their studies, their readings, and their written responses—both guided and freeform.  Last night’s assignment was a free-form genre response to the entire ~60-page notebook, with required parent involvement. My son decided to do the assignment in the form of a written quiz. He gave me five minutes to read through his entire notebook (towards the end of the time he relented and granted me an additional three minutes) and afterwards I was to take his quiz. Picture me holding my usual pre-bedtime court, with as much evening concentration as I can muster focused on my son’s notebook, and add the son bouncing around on the bed the whole time. Sidenote to people with younger children: the arrival of puberty does not mean that the kids stop bouncing on the beds. At least not in our household.

And then the timer was up, the notebook was snatched away, and I was handed a sheet of paper with scrawled questions:

This interactive notebook assignment was on which page?
 a. 60
b. 59
c. 62
d. 56”

Yikes.

The second question:
How many wrong answers did I get on my literary methods quiz?
a. -11
b.  -7
c. -1
d. 100%

Oh no. A trick question. My son is hovering intently as my pen decides between answer c. and d.

And a bit better, but still I got it wrong:
The plotline summary was written about which book I read?
a. Wonder
b. Artemis Fowl
c. From the Mixed-Up Files of Mrs. Basil E. Frankweiler
d. UnLunDun

And by this time I’m reflecting on the skill that is required to construct really good quiz questions. The way that the question is written can render it trivial to completely opaque. The goal is to ask the “just right” question: that tests concepts but is not open to too many interpretations. Bonus if the question does not take too much time. Especially in a 1-hour classroom format, I do not want to give quizzes that solely test a student speed. We do work at a variety of different paces.

So—take some short-format examples from plate tectonics classes:

The following observations provided evidence for continental drift except
a.           Mountain Fold Belts
b.           Geologic Age provinces
c.           Igneous provinces
d.           Stratigraphic sections
e.           Paleoclimate evidence such as evaporates or glacial deposits
f.           Stable isotope fractionation

This example is too easy. The first five answers come right out of the first few paragraphs of one of our plate tectonics textbooks. The last answer is completely from left field in the context of undergraduate plate tectonics. Most UCLA students are good test takers, and would see right through this question.

Here is a rewritten version:
The following types of observations provided evidence for continental drift except
a.           Geologic evidence such as mountain belts
b.           Paleoclimate evidence such as glacial deposits or evaporite beds
c.           Paleontologic evidence such as Glossopteris and Gangamopteris
d.           Seismology--information from earthquake travel times

I like this question better for a few reasons: 1. The answers to the questions provide internal explanations and definitions, to reinforce learned concepts. 2. All of the answers are covered in detail in the readings and the lectures. For a person who has not studied the history of plate tectonics a little bit, it is not clear which is the correct answer.  I won’t give it away here—so please put your answers in the comments!

Here is a true/false question:

True or false: 
The force of the moon on the Earth (tidal forcing) provides the mechanism underlying continental drift

Again, Too easy.

Here is a better way to ask a true/false
For the following question, answer true or false. If the statement is false, change the underlined part of the statement to make it true
The force of the moon on the Earth (tidal forcing) is the best explanation for continental drift

This works, but is a bit easy. There is some skill involved in choosing which are the best words to underline in the statement. Here is a better way:

The force of the moon on the Earth (tidal forcing) is the best explanation for continental drift

Now this question will force students to think a bit, and the answers can be graded on a broad continuum.

The ideal test question for college students:
1.     tests concepts and facts together
2.     is not too trivial but not too complex
3.     can be finished by most students with plenty of time to think
4.     Teaches and reinforces the course ideas and concepts

The ease or difficulty of an exam has as much to do with the test-writer as it has to do with the test-taker.  

Do you have any examples of great exam questions? Terrible ones? If you prepare exams, have you had any training in the skills of test preparation? Do you find test-taking and/or test preparing fun? Frustrating?

2 comments:

  1. Valuable site, where did u come up with the information in this posting? I am pleased I discovered it though, ill be checking back soon to find out what new content pieces u have.
    glass door lock

    ReplyDelete
  2. Keep it up!! You have done the nice job having provided the latest information. garage door experts abc

    ReplyDelete