The Role of Lexical Collocations and Learner and Course Variables in Determining Writing Quality in Assignments from a First Year Composition Programme

Abstract

Over the last decade, studies have shown how the diverse and sophisticated use of collocations in student writing has a relationship with grades awarded by raters. However, these studies have a number of contextual and methodological limitations. Contextually, studies have largely been limited to second language writers whose writing has been evaluated according to the Common European Framework of Reference (CEFR). Methodologically, these studies have used a small number of linguistic measures of sophistication in the form of association measures. Such a limitation is a result of the guiding measurement literature appearing in a fragmented manner. This fragmented manner does not make bringing together the range of possible measures we may use an easy task. Further still, the relationship between collocations and writing quality has been studied using correlation and/or multiple linear regression techniques. These techniques do not consider the effect that the hierarchical nature of the corpus may have on measuring this relationship. This means previous studies have not accounted for the random effect of individual writers or raters in their analyses. The present study tackles these limitations. The study adds contextual weight to the literature by investigating the relationships between collocations and writing quality in a corpus of First Year Composition (FYC) essays written by speakers of English as a first and second language in the US. To tackle the methodological limitations, the research is divided into two studies. The first study examines the theoretical and mathematical connections between a subset of association measures via the use of a cluster analysis. From this cluster analysis a case is made as to which measures should be selected for inclusion in the subsequent investigation of the collocation-writing quality relationship. The second study also tackles methodological limitations by modelling the relationship between collocations and writing quality through a mixed effects logistic regression model which accounts for random writer and rater effects. Considered individually, each study has implications for how we measure the relationship between collocations and writing quality. Study one has implications for discussions around measure use and how we understand the properties of collocation. Study two has implications for understanding the relationship collocations have with the grades writers are awarded on the FYC programme. Taken together both studies allow researchers to consider how such relationships may be measured in the future so as to appreciate a broader range of contexts, appropriate objective measure selection and rigorous statistical techniques.