Can questions be pre-tested?

It is advisable to pre-test questions before they are incorporated into examinations and/or question banks.

A pre-test of questions would normally be taken by students who are representative of those who will be taking the actual assessment. Ask students to record their start and finish times. (If using CBA, this can be done automatically.) Following the pre-test, analyse the facility and discrimination of the individual items. You might also wish to analyse the frequency with which distracters within items are chosen and replace those which are selected infrequently with more plausible options.

Analysis of the pre-test may also provide useful feedback on the module or course itself, by highlighting areas of difficulty.

One strategy for pre-testing would be to use questions one year as part of a formative assessment the score of which would not effect the student's module mark. After analysing and modifying the questions, they could be used the following year as part of a summative assessment.

Beta-testing questions is another technique which can be used once you engage in a regular programme of objective testing. Include new questions in an assessment with known "good" questions. After the students have taken the assessment, evaluate the facility and discrimination of these questions. If these indicators suggest that the questions are good, then leave them as part of the assessment. If not, omit them from the calculations of scores.