A study of raters’ background knowledge, rater training and other factors affecting their decision making in rating Thai plots’ English speaking proficienc

Abstract

The objectives of this study were to examine two kinds of raters having different background knowledge i.e. linguistic and operational raters with and without rater training experience when they assessed Thai pilots’ English language speaking performances on RELTA, and to explore the other factors affecting their decisionmaking in awarding the scores to the candidates. The participants in the study were 20 raters. They were categorized into two main groups, linguistic and operational, based on their educational and professional background. Then, they were divided further into four sub-groups i.e. linguistic/trained, linguistic/untrained, operational/trained, and operational/untrained. The subjects participating in the main study were purposively selected for data analysis. The source of data was the RMIT English Language Test for Aviation (RELTA). The instruments were questionnaires, rater score sheet and remarks, and semi-structured interviews. The research design was the quasiexperimental research. The 2x2 ANOVA and t-test were used to analyze the quantitative data, and the content analysis was used to analyze the qualitative one, and to confirm the results obtained from the quantitative analysis.