• Home
    • About ICAS Assessments
    • About Our Team
    • Blog
  • Assessment Products
    • ICAS
      • Medal Ceremony
    • Reach
    • Scout
  • Assessment Calendar
  • Event Support
    • Technical Requirements
    • Locked Down Browser
    • Assessment Event Forms
  • Portals
    • Assessments Portal
    • Results Portal
    • Shop
      • Toll-Free Australia: 1800 931 775
    • CONTACT US
      BUY NOW
    ICAS AssessmentsICAS Assessments
    • Home
      • About ICAS Assessments
      • About Our Team
      • Blog
    • Assessment Products
      • ICAS
        • Medal Ceremony
      • Reach
      • Scout
    • Assessment Calendar
    • Event Support
      • Technical Requirements
      • Locked Down Browser
      • Assessment Event Forms
    • Portals
      • Assessments Portal
      • Results Portal
      • Shop
      • Toll-Free Australia: 1800 931 775
    • CONTACT US
      BUY NOW

      Blog

      • Home
      • Blog
      • Part 2 – Writing Tests. The Science of Writing Tests

      Part 2 – Writing Tests. The Science of Writing Tests

      • Categories Blog
      • Date January 20, 2016

      In the world of assessment, tests are often called “instruments”. It is a bit of a pretentious term but expresses the idea that there is a purpose to every test. Each test is an instrument intended to gather information about what is going on in people’s minds. There is art in every test question but there is science behind every test.

      That science has a long (and sometimes very dark) history. The desire to know how people think has been motivated by both good and bad motives. The underlying concepts remain the same.

      At a basic level is the concept of reliability. A reliable test is one that produces test scores that are consistent. One half of a test (e.g. all the even numbered questions) should correlate with the other half (e.g. all the odd numbered questions). A test that contains wildly different subjects or skills is not going to produce reliable data. There are various statistical methods that can be employed to quantify how reliable a set of test data is.

      However, a reliable test doesn’t need to be correct or appropriate. A test full of wrong questions can still be reliably wrong. Reliability is a powerful concept but it isn’t enough.

      A broader and deeper idea is validity. A simple way of thinking about validity is asking whether the data from a test is really telling you what it claims to be telling you. Practically, test validity is about collecting evidence to support the claims we might make about a test. Is a test fit for purpose? Is it really testing the skills or knowledge it claims to be testing? Does the data have any hidden biases? Is the test valid for all the people it is being used with?

      In practice, validity is always an open question for any test. There are multiple perspectives of validity including:

      • Predictive validity: how well does a test predict future performance in the same area.
      • Face validity: do expert reviewers in the subject area agree that the test content is correct and appropriate.
      • Consequential validity: how will the test data be used and what decisions will be made using the test data.

      These different concepts are complimentary but they each suggest different approaches to collecting evidence to show a test is valid. Some involve review by experts and others involve statistical analysis and other require long-term ongoing research.

      It is a rich and complex field, with few easy answers.

      Tag:Designer, SEO

        Previous post

        Classroom Tips. Running Online Assessments
        January 20, 2016

        Next post

        Part 1 – Writing Tests. The Art of Writing Tests
        December 3, 2020

        You may also like

        art-of-writing-tests
        Part 1 – Writing Tests. The Art of Writing Tests
        3 December, 2020
        classroom-tips
        Classroom Tips. Running Online Assessments
        20 October, 2015
        hidden-secret-of-multiple-choice
        Hidden Secret of Multiple Choice
        20 October, 2015

        Search

        Categories

        • Blog

        Our Products

        ICAS

        ICAS

        Scout

        Scout

        Reach

        Reach

        +61 2 8267 8800 Toll-Free Aus: 1800 931 775 Toll-Free NZ: 0800 440 904
        [email protected]
        80 Bay Street, Ultimo NSW 2007 Australia
        • Facebook
        • Home
          • About ICAS Assessments
          • About our Team
          • Blog
        • Assessment Calendar
        • Assessment Products
          • ICAS
            • – Medal Ceremony
          • Reach
          • Scout
        • Event Support
          • ICAS Support
          • Reach Support
          • Scout Support
          • Technical Requirements
          • Locked Down Browser
          • Assessment Event Forms
        • Portals
          • Assessments Portal
          • Results Portal
          • Shop

        Our Mission: World class assessments driving student success.

        Copyright © 2020 ICAS Assessment. All rights reserved. Privacy Policy | Terms & Conditions