Evaluation of Scientific Research Assignments with Multidimensional Rasch Measurement Model

Abstract

The purpose of this study is to investigate the usefulness of the many-facet Rasch model (MFRM) in evaluating the quality of performance related to preparing Research assignment in higher education. The Rasch Model utilizes item response theory stating that the probability of a correct response to a test item/task depends largely on a single parameter, the ability of the person. MFRM extends this one-parameter model to other facets, for example, rater severity, rating scale format, task difficulty levels. This paper specifically investigated research preparation ability in terms of items/task difficulty and rater severity/leniency. Fourth year counseling students prepared research assignments during the autumn semester of the 2009-2010 school years in the "Research Methods in Education" course. Six judges evaluated each students assignments using "Research Assignment Evaluation Rubric". The results of this study demonstrated that the MFRM is a powerful tool for handling polytomous data in performance and peer assessment in higher education.