Abstract

Aim: To evaluate the difference/bias rate between open-ended (OE) vs multiple-choice questions (MCQs) for a critical nursing skill.

Method: Three hundred and seventy-six nurses responded from 3 nursing schools to a 20-item MCQs and OE instrument. Questions concerned nursing knowledge of maintaining clear airways and interventions related to trachea suction. Subjects first responded to the OE instrument followed by MCQs. Both tests were paper based. Statistical analyses included paired t-tests and one-way ANOVA.

Result: Outcomes showed significant differences between OE and MCQs, MCQs were scored higher. On MCQs, Licensed Vocational Nurse (LVN) and Associate Degree in Nursing (ADN) group scores did not differ. As for OE, both ADNs and BSNs did significantly better than LVNs. BSNs increased their lead markedly over both groups in OE.

Conclusion: MCQs overestimated knowledge levels when respondents’ knowledge base was less confident or professional qualification was lower. When knowledge base was solid, the difference between OE and MCQs disappeared.

Highlights

A. Multiple-choice test scores can be biased upwards compared to open-ended tests

B. BSN nurses outperformed LVN and ADN nurses on open-ended tests

C. Nurses with more/longer experience score equal on both test types

D. Open-ended tests are still advantageous to assess critical skills

Keywords: Open-ended; Multiple-choice; Test; Test bias; Critical skill