Abstract
Starting from the influential study by Paquot (2019), many scholars have conceptualized phraseological complexity as composed by two intertwined components: phraseological diversity and sophistication (e.g., Vandeweerd et al., 2022). To compute phraseological diversity, lexical diversity measures such as root type-token ratio have been used. Although there have been significant contributions to the debate on the validity of such indices (Kyle et al., 2021; McCarthy & Jarvis, 2010), a thorough comparison with theoretical observations from other fields of research would greatly enhance our understanding of the specific type of complexity that these measures truly capture.
Typological literature has distinguished two types of complexity (Dahl, 2009): Kolmogorov complexity refers to the length of the shortest description needed to represent a string of symbols. Gell-Mann effective complexity is understood as the length of description required to specify the set of regularities present in a string. According to Kolmogorov complexity, a string that lacks any regular patterns or structure would require a very long thus complex description, as each element of the string would need to be individually accounted for. From the perspective of Gell-Mann effective complexity, instead, a string without any regularities would have no complexity, since it contains no structured patterns.
In ecology (Parrott, 2010), a clear-cut distinction is made between measures that operationalize the Kolmogorov complexity (“Type 1 measures”) vs. the Gell-Mann effective complexity (“Type 2 measures”). Type 1 measures favor random sequences and can be visualized as linear functions on the continuum between order and disorder; type 2 measures, instead, can be visualized as a convex function that reaches its peak when a system strikes a balance between rules and exceptions (Fig. 1). Intuitively, the latter might be the kind of complexity we want to be able to measure in a text.
This contribution will therefore try to answer the following research question: which type of complexity (Kolmogorov vs. Gell-Mann) is measured by the most commonly used lexical diversity indices?
In addressing this question, we will conduct a simulation to evaluate lexical diversity in Italian texts employing TTR-based metrics along with entropy-based ones (as found in Garner, 2020), including Type 2 measures, such as fluctuation complexity, as described in Parrot (2010). Each measure will be calculated varying the text length by truncation to study the length dependency of its behavior. Additionally, each measure will be calculated on texts which have different expected behaviors in terms of Kolmogorov and Gell-Mann complexity: real texts – expert-authored (e.g., Repubblica corpus) and non-expert-authored (e.g., LEONIDE, ITACA) – are expected to display a certain degree of regularity, thus being representative of a medium to high Gell-Mann complexity; synthetic texts sampling the real texts using either the original word distributions or a uniform distribution (i.e., every word has an equal probability of appearing), instead, do not show this regularity, thus representing a medium to high Kolmogorov complexity, but a low Gell-Mann complexity. We will then be able to distinguish between type 1 and type 2 measures by identifying the set of texts (real or synthetic) for which the various lexical diversity measures yield the highest values. Results will contribute to the assessment of metrics’ validity for different conceptualizations of complexity.
References:
Dahl, Ö. (2009). Testing the assumption of complexity invariance: The case of Elfdalian and Swedish. In G. Sampson, D. Gil, & P. Trudgill (Eds.), Language Complexity As an Evolving Variable (pp. 50-63). Oxford University Press.
Garner, J. (2020). The cross-sectional development of verb–noun collocations as constructions in L2 writing. International Review of Applied Linguistics in Language Teaching, 60, 909-935.
Kyle, K., Crossley, S. A., & Jarvis, S. (2021). Assessing the Validity of Lexical Diversity Indices Using Direct Judgements. Language Assessment Quarterly, 18(2), 154-170.
McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381–392.
Paquot, M. (2019). The phraseological dimension in interlanguage complexity research. Second Language Research, 35, 121-145.
Parrott, L. (2010). Measuring ecological complexity. Ecological Indicators, 10(6), 1069-1076.
Vandeweerd, N., Housen, A., & Paquot, M. (2022). Comparing the longitudinal development of phraseological complexity across oral and written tasks. Studies in Second Language Acquisition, 1-25.