Examining Scale Items in Terms of Method Effects Based on the Bifactor Item Response Theory Model

Abstract

Purpose: The current study aims to apply a one-dimensional (the graded response model) and a multidimensional (the bifactormodel) item response theory model to evaluate the presence of method effects on the data obtaining from the administrationof the scale measuring emotional school engagement of students as a part of 2015 Programme for International StudentAssessment (PISA).Design/Methodology/Approach: The model data fits of the one-dimensional and bifactor model conceptualizations werecompared on the data obtained from a large sample of Turkey school children.Findings: The item parameters and model data fit statistics provided evidences for that the school engagement scale of thePISA 2015 measures a primary construct (the emotional school engagement) with two nuisance factors (the negative andpositive wording effects).Highlights: The results of the present study supported the use of bifactor model in evaluating the presence of method effects.Therefore, researchers using the Emotional School Engagement Scale of PISA are recommended to utilize more sophisticatedstatistical techniques such as the bifactor item response theory model.