Adaptation of "Objective Structured Assessment of Technical Skills" for Adenotonsillectomy into Turkish: A Validity and Reliability Study

Abstract

Objective: To adapt an objective surgical assessment tool (OSATS) for adenotonsillectomy/tonsillectomy to Turkish.

Methods: A prospective longitudinal validation study was planned. Adenoidectomy items were added to the tonsillectomy OSATS. Data were collected from 80 real-time observed surgical performances. To examine interrater reliability, 25 performances were simultaneously scored by two evaluators. In total, 105 test forms were filled, and reliability and validity studies were conducted.

Results: Regarding reliability, Cronbach's alpha values were found to be >0.95, indicating very high internal consistency. For interrater reliability, a statistically significant positive correlation was found between the two evaluators' task-specific checklist (TSC) (p=0.0001; r=0.969) and global rating checklist (GRC) (p=0.0001; r=0.957). Between the TSC and GRC, there was a statistically significant positive correlation (p=0.0001; r=0.951). A model was developed using the generalizability theory, and the highest variances belonged to surgical processes. Regarding validity, the trainers' scores were significantly higher than the trainees' scores (p=0.0001). As surgical experience increased, both the TSC and the GRC scores significantly increased (p=0.0001). A trainee's first-ever 34 adenotonsillectomy performances were scored. A scatter plot of the obtained data showed that the trainee had gained more scores as the number of operations increased.

Conclusion: The results indicate that adenotonsillectomy OSATS is a reliable and valid tool for the assessment of residents' surgical skills. This is the first OSATS developed for adenotonsillectomy in the Turkish literature. It is ready to be used in the operating room to provide constructive feedback and monitor the development of trainees' surgical skills in adenotonsillectomy.

C1 [Kara, Cuneyt Orhan; Mengi, Erdem; Tumkaya, Funda; Ardic, Fazil Necdet] Pamukkale Univ, Dept Otorhinolaryngol, Sch Med, Denizli, Turkey.

[Senol, Hande] Pamukkale Univ, Dept Biostat, Sch Med, Denizli, Turkey.