Scoring Methods for Multiple Choice Tests: How does the Item Difficulty Weighted Scoring Change Student’s Test Results?

Abstract

The past studies on weighted test scoring were focused on the correlation with unweighted test scores. Yet, it is important to investigate the effect of weighted scoring on students’ pass and fail rates. In the present study, it was aimed to compare students’ test scores, item and test statistics calculated based on the unweighted (1 – 0) and item difficulty weighted scores (Qj – 0). The study also included a proposal for converting the weighted scores into a 100-point scale system. A teacher-made 34-item multiple-choice achievement test was conducted to a group of 431 people via learning management system. As a result of the data analysis, the McDonald's Omega internal consistency coefficients that were obtained according to the 1 – 0 and (Qj – 0) methods were obtained as .725 and .721, respectively. The Pearson’s product moment correlation coefficient was .916, and the Spearman’s rank-order correlation coefficient was .926 between student scores obtained according to the two methods. Furthermore, a criterion-based evaluation was made based on the two criteria (test scores of 50 and 60), and the numbers of the students who were successful and unsuccessful in the course were determined according to both scoring methods. Accordingly, it was found that more students would be considered unsuccessful in the course in the (Qj-0) scoring method; however, it was understood that this method could reveal differences among individuals more than the unweighted scoring method.