Analyses for a vocabulary profiler data

Question

I am writing a paper which uses data from vocabulary profiler (a tool that tokenizes and analyzes a text, output is something like "This text consists of 20% of Level X vocabulary, 20% of Level Y vocabulary, and 50% of Level Z vocabulary."). To do this a profiler references a list of words separated by level.

My data points are a collected using a corpus of 41 texts, for each of which I collected the following metrics: (1) vocabulary percentages, each text gets 7 numbers (since 7 levels of vocabulary), (2) levels of the "text complexity" as predicted by three different GPTs. This text complexity comes on the scale 1-7 and corresponds to levels of vocabulary I profiled for.

What I want to show or, rather, test, is whether there is a correlation (or one can be predicted by the other) between the levels percentages as profiled by me and the "level of complexity" of the text as determined by the three GPTs. I also controlled for temperature (three settings: .1, .5, .9).

I am hesitant how to run a regression in this case and whether to run it. Should I treat levels as a continuous variable? Levels are 1-7, but there's nothing to suggest the "distance" between Level 1 and Level 2 is the same as the distance between Level 2 and Level 3. Levels actually mean to reflect levels of complexity of the text, or in my case levels of language proficiency.

I could run a dozen of regressions, but I am interested in the interaction of the factors I controlled for, so if anyone could suggest some options for other analyses, I'd be very grateful!

Thanks in advance!

Answer 1

What you are describing is an ordinal value:

• Nominal: Values are categorical and there is no logical monotonic ordering (e.g., patients' eye color)
• Ordinal: Values are categorical, but there is a logical monotonic ordering (i.e., $1>2>\dots>7$ in your case). However, the distance between $1$ and $2$ does not need to be equally large as the distance between $2$ and $3$, and so on.
• Ratio scale: Values can be interpreted as numeric, and differences can be compared meaningfully using division (e.g., "it is $1^\circ \text{C}$ warmer than yesterday" is understood without knowing yesterday's temperature).

You can analyse ordinal data in many ways, like with ordinal regression. However, from your question, it seems more appropriate to consider the 7 levels not as the outcome, but as an explanatory variable. There is a great discussion of that over here.