• Home
    • About ICAS Assessments
    • About our team
    • International customers
  • Assessment products
    • ICAS
      • ICAS past papers
      • ICAS medal ceremony
    • Reach
    • Scout
  • Assessments calendar
    • Australia
    • New Zealand
    • Pacific & International
  • Blog
  • Event support
    • ICAS support
      • ICAS FAQs – schools
    • Reach support
      • Reach FAQs – schools
    • Scout support
      • Scout FAQs – schools
    • Technical requirements
    • Locked-down browser
    • Assessment Event forms
    • Live Assessment Events status
  • Portal
    • Assessments portal
    • Results portal
    • Shop
      • Australia Toll-Free: 1800 931 775
    • Contact us
      Buy now
    ICAS Assessments – World Class School Assessments & Academic CompetitionsICAS Assessments – World Class School Assessments & Academic Competitions
    • Home
      • About ICAS Assessments
      • About our team
      • International customers
    • Assessment products
      • ICAS
        • ICAS past papers
        • ICAS medal ceremony
      • Reach
      • Scout
    • Assessments calendar
      • Australia
      • New Zealand
      • Pacific & International
    • Blog
    • Event support
      • ICAS support
        • ICAS FAQs – schools
      • Reach support
        • Reach FAQs – schools
      • Scout support
        • Scout FAQs – schools
      • Technical requirements
      • Locked-down browser
      • Assessment Event forms
      • Live Assessment Events status
    • Portal
      • Assessments portal
      • Results portal
      • Shop
      • Australia Toll-Free: 1800 931 775
    • Contact us
      Buy now

      Blog

      • Home
      • Blog
      • Hidden secret of multiple choice questions

      Hidden secret of multiple choice questions

      • Categories Blog
      • Date October 20, 2015
      hidden-secret-of-multiple-choice

      One of the exciting changes in my work over the past few years has been the wide range of new question types that are available in online tests. There are so many more ways for students to interact with assessments and new ways for them to demonstrate their skills and knowledge.

      Despite this, there is still a big place for the humble multiple-choice question. They’ve not always had the best reputation and they are often perceived as being only suitable for testing shallow concepts and as being prone to guessing. However, a well-constructed multiple-choice question has a lot of power and subtlety and can be a very effective way of surveying what students know and can do.

      What is the secret of a good multiple-choice question? It all comes down to the options. The options consist of one correct response which we call the ‘key’, and a number of incorrect responses that we call ‘distractors’. At a basic level, the job of the distractors is to hide the key but if that is all they do then you can end up with a test question that genuinely is shallow and prone to guessing. Back when I was a maths teacher (a long time ago now) I’d see multiple-choice questions that were written with distractors that were three numbers similar in size to the key. This is sometimes characterised as one-below, one-above and one-very close. This approach is very inadequate for several reasons! The most obvious problem is that it encourages students to guess that the answer is somewhere in the middle of the range of numbers!

      A more sophisticated method of distractor choice is to consider the likely errors students will make. Not only does this provide meaningful distractors, it is also a useful way to consider the quality of the item in general.  Identifying the common errors involves a degree of guess work but can be made less subjective by examining the performance of similar items in previous tests. A knowledge of the skills and knowledge of the test population is also helpful.

      One approach to identifying common errors in a maths question is to look at the steps needed to find the key. For example, the addition 27 + 26 requires the student to add several digits together whilst being aware of place value. The most common error would be to treat 7 + 6 as 3 rather than 13. If students were asked this problem as a free-response question, the most common incorrect answer would be 43.  As we need three plausible distractors, the task in the item has to have enough complexity to generate three errors. The addition 27 + 26 would be inadequate for most audiences. More complex problems may generate so many plausible errors that selecting the best three can be difficult.

      Using likely errors for distractors also provides rich information. Each distractor is essentially a code for a specific mistake. That makes it easier to see what kind of errors a large number of students are making. This is a very powerful feature of multiple-choice question and yet it is often overlooked. Few other forms of assessment so neatly enable a teacher to see the kinds of mistakes groups of students are making.

      So while I definitely enjoy the bigger range of choices we now have for creating assessments, I still have a place for the humble and oft-maligned multiple-choice question.

      Tag:WordPress

        Next post

        Part 2 - Writing tests. The science of writing tests
        January 20, 2016

        You may also like

        Free ICAS Past Papers: Why Is There Not Such A Thing
        Free ICAS past papers: Why is there not such a thing: A behind-the-scenes look at how we write ICAS questions
        27 May, 2022
        ICAS testimonials blog post Nancy
        “Keeps me on my toes”: teachers, parents and students tell us why they love ICAS
        26 May, 2022
        10 school competition benefits blog post
        Top 10 benefits of school competitions
        10 May, 2022

        Search

        Our Products

        ICAS

        ICAS

        Scout

        Scout

        Reach

        Reach

        • Facebook
        • Youtube
        • Home
          • About ICAS Assessments
          • About our team
          • International customers
          • Blog
        • ICAS, Reach and Scout calendar
        • Assessment products
          • ICAS
            • – ICAS past papers
            • – Medal ceremony
          • Reach
          • Scout
        • Event support
          • ICAS support
          • Reach support
          • Scout support
          • Technical requirements
          • Locked-down browser
          • Assessment Event forms
        • Portals
          • Assessments portal
          • Results portal
          • Shop

        Our Mission: World-class assessments driving student success.

        Copyright © 2022 ICAS Assessments. All rights reserved. Privacy Policy | Terms & Conditions | Refund Policy