Writing is one of the key components of language proficiency. Every year in Sweden, upper secondary school students are required to take the national tests of English, which are created by the Swedish National Agency for Education (SNAE) and intended to establish to what extent the students’ proficiency is in line with the course expectations. This paper aims to study the role that lexical complexity measures play in the grading of such texts written by Swedish upper-secondary school students. The data consist of graded example texts (n=142) provided by SNAE to teachers in the assessment instructions to illustrate how the tests are to be assessed and a corpus consisting of student texts graded by teachers (n=190). The assessment instructions indicate that there should be a cline from the lowest to highest grade in terms of lexical and phraseological complexity. Therefore, a wide range of lexical (e.g., word frequency, dispersion and diversity) and phraseological measures (e.g., n-gram register and association strength) were applied to discern if a sequential distribution between different grades exists. Preliminary results show that very few complexity features predict grade in any meaningful way, suggesting that lexical complexity is largely overlooked in the assessment of national tests in Sweden. The SNAE and practicing teachers also appear to value different aspects of lexical complexity, which may have consequences for their classroom practice and the development of students’ writing proficiency. This discrepancy and its possible implications are further discussed in relation to ensuring a fair and reliable assessment practice.