• Home
    • About ICAS Assessments
    • About our team
    • International customers
  • Assessment products
    • ICAS
      • ICAS past papers
      • ICAS medal ceremony
    • Reach
    • Scout
  • Assessments calendar
    • Australia
    • New Zealand
    • Pacific & International
  • Blog
  • Event support
    • ICAS support
      • ICAS FAQs – schools
    • Reach support
      • Reach FAQs – schools
    • Scout support
      • Scout FAQs – schools
    • Technical requirements
    • Locked-down browser
    • Assessment Event forms
    • Live Assessment Events status
  • Portal
    • Assessments portal
    • Results portal
    • Shop
      • Australia Toll-Free: 1800 931 775
    • Contact us
      Buy now
    ICAS Assessments – World Class School Assessments & Academic CompetitionsICAS Assessments – World Class School Assessments & Academic Competitions
    • Home
      • About ICAS Assessments
      • About our team
      • International customers
    • Assessment products
      • ICAS
        • ICAS past papers
        • ICAS medal ceremony
      • Reach
      • Scout
    • Assessments calendar
      • Australia
      • New Zealand
      • Pacific & International
    • Blog
    • Event support
      • ICAS support
        • ICAS FAQs – schools
      • Reach support
        • Reach FAQs – schools
      • Scout support
        • Scout FAQs – schools
      • Technical requirements
      • Locked-down browser
      • Assessment Event forms
      • Live Assessment Events status
    • Portal
      • Assessments portal
      • Results portal
      • Shop
      • Australia Toll-Free: 1800 931 775
    • Contact us
      Buy now

      Blog

      • Home
      • Blog
      • Part 2 – Writing tests. The science of writing tests

      Part 2 – Writing tests. The science of writing tests

      • Categories Blog
      • Date January 20, 2016
      science-of-writing-tests

      In the world of assessment, tests are often called “instruments”. It is a bit of a pretentious term but expresses the idea that there is a purpose to every test. Each test is an instrument intended to gather information about what is going on in people’s minds. There is art in every test question but there is science behind every test.

      That science has a long (and sometimes very dark) history. The desire to know how people think has been motivated by both good and bad motives. The underlying concepts remain the same.

      At a basic level is the concept of reliability. A reliable test is one that produces test scores that are consistent. One half of a test (e.g. all the even numbered questions) should correlate with the other half (e.g. all the odd numbered questions). A test that contains wildly different subjects or skills is not going to produce reliable data. There are various statistical methods that can be employed to quantify how reliable a set of test data is.

      However, a reliable test doesn’t need to be correct or appropriate. A test full of wrong questions can still be reliably wrong. Reliability is a powerful concept but it isn’t enough.

      A broader and deeper idea is validity. A simple way of thinking about validity is asking whether the data from a test is really telling you what it claims to be telling you. Practically, test validity is about collecting evidence to support the claims we might make about a test. Is a test fit for purpose? Is it really testing the skills or knowledge it claims to be testing? Does the data have any hidden biases? Is the test valid for all the people it is being used with?

      In practice, validity is always an open question for any test. There are multiple perspectives of validity including:

      • Predictive validity: how well does a test predict future performance in the same area.
      • Face validity: do expert reviewers in the subject area agree that the test content is correct and appropriate.
      • Consequential validity: how will the test data be used and what decisions will be made using the test data.

      These different concepts are complimentary but they each suggest different approaches to collecting evidence to show a test is valid. Some involve review by experts and others involve statistical analysis and other require long-term ongoing research.

      It is a rich and complex field, with few easy answers.

      Tag:Designer, SEO

        Previous post

        Hidden secret of multiple choice questions
        January 20, 2016

        Next post

        Part 1 – Writing tests. The art of writing tests
        December 3, 2020

        You may also like

        Free ICAS Past Papers: Why Is There Not Such A Thing
        Free ICAS past papers: Why is there not such a thing: A behind-the-scenes look at how we write ICAS questions
        27 May, 2022
        ICAS testimonials blog post Nancy
        “Keeps me on my toes”: teachers, parents and students tell us why they love ICAS
        26 May, 2022
        10 school competition benefits blog post
        Top 10 benefits of school competitions
        10 May, 2022

        Search

        Our Products

        ICAS

        ICAS

        Scout

        Scout

        Reach

        Reach

        • Facebook
        • Youtube
        • Home
          • About ICAS Assessments
          • About our team
          • International customers
          • Blog
        • ICAS, Reach and Scout calendar
        • Assessment products
          • ICAS
            • – ICAS past papers
            • – Medal ceremony
          • Reach
          • Scout
        • Event support
          • ICAS support
          • Reach support
          • Scout support
          • Technical requirements
          • Locked-down browser
          • Assessment Event forms
        • Portals
          • Assessments portal
          • Results portal
          • Shop

        Our Mission: World-class assessments driving student success.

        Copyright © 2022 ICAS Assessments. All rights reserved. Privacy Policy | Terms & Conditions | Refund Policy