Part 1 – Writing tests. The art of writing tests

It was many years ago and on the other side of the world. I was a maths teacher then in South London and a student was looking at a sample test paper for the national mathematics tests.

“But where do they come from?” they asked.

“I guess some people write them,” I said, unwilling to say directly that I actually had no idea and indeed, had never paid much thought to where tests come from. “Maybe,” I added, “gnomes write them. Test gnomes working in huge caves underground.”

Years later, through a series of unplanned journeys and events I ended up becoming one of those self-same test gnomes.

There is an art to writing good test questions. A good test question is short but long enough to convey everything a student needs to know to answer the question. They need to use simple language but complex enough to express subtle and technical ideas. They need to avoid irrelevant detail but also be colourful and engaging. Doing all of these things in a set of brief sentences is a challenge.

Most of the test writers I have worked with have been teachers or former teachers. It is a natural move from educating to testing but it isn’t always an easy move. As a teacher you can always clarify what you just said to your class. If your words are met by blank incomprehension, you can try again with a different way of explaining the same concept. With a test question there is no second chance and no scope to clarify. The art of writing a test question is the art of asking something subtle with few words and extreme clarity.

So much of the work isn’t the initial question but long amounts of review and refinement. By the time a test question has made its way into a test and to a student, it has been written and re-written by multiple people, multiple times.

And that is the hidden work that goes on behind every test.