Can the quality of individual questions be monitored?

Yes, one of the advantages of using objective tests is that a variety of analyses can be run to determine how well individual questions (and parts of questions) perform. This information enables the lecturer to improve or eliminate weak questions (such as those which are too easy or difficult).

Additionally, the performance of the test-takers (as a cohort and as individuals) can be analysed and this information can help identify general problem areas within the module, students who are struggling and students who are not being sufficiently challenged.

While all of these analyses can be done by hand, some CAA software packages include a reporting facility which generates a range of statistical analyses on the performance of both the test-takers and the questions.

The following are the types of statistical measures which help identify the usefulness of a particular question.

• discrimination
• facility
• frequency analysis