2
$\begingroup$

This post is related to my previous ones, but now I'm looking at each year separately (i.e, this is not a repeated measures design). My data set looks like this:

SUBJECT    PROFICIENCY (in Eng) SCORE LANGUAGE

PETER      100                   154    Spanish 
PETER      100                   132    English
MARY       95                    191    Spanish
MARY       95                    139    English

So each SUBJECT was tested twice: one test in English and one test in Spanish. Therefore, each SUBJECT has 2 exam scores. I want to see the impact of Language on the test scores. However, I already know that Proficiency in English is highly correlated to the test score in English:

correlation PROFICIENCY and the ENglish Exam Score

What I'm wondering is if I can fit a model in which I could use proficiency as a "moderator" (something like a partial correlation). My idea was to fit a model with a random intercept for Proficiency. The thing is, I only have one Proficiency score per student (I don't have a Proficiency score for Spanish). Hence, I have two entries for Language and one for Proficiency (in Eng).

Would this make sense:

mod1 <- lmer(ExamScore ~ LANGUAGE + (1 | SUBJECT) + (1|PROFICIENCY IN ENG), data = data)

I have seen similar questions elsewhere with a categorical moderator (here and others

Thanks in advance, any thoughts would be much appreciated :)

Improve this question
$\endgroup$
7
  • $\begingroup$ Can you explain educational setting: Is English the native language? Or "first" foreign language while Spanish -- the second? I'm wondering whether a model like Spanish_score ~ English_score + English_proficiency might not be interesting / applicable. $\endgroup$
    – dipetkov
    Aug 28, 2022 at 12:17
  • $\begingroup$ @dipetkov , so English is the language that they're learning (ie, the second), Spanish is their native language (that's why I don't have a proficiency score for Spanish). The ideia is to see whether the different languages play a role on their scores on their exams. So I had a wide table and turned it into a long table in order to do ExamScore ~ Language, but then I noticed that proficiency in English is correlated with the ExamScore, so I guess that I'm supposed to control its effect on the exam $\endgroup$
    – Larissa Cury
    Aug 28, 2022 at 14:52
  • $\begingroup$ because subjects with a higher Proficiency will likely have a higher score in English (ps: thank you once more!) $\endgroup$
    – Larissa Cury
    Aug 28, 2022 at 14:53
  • $\begingroup$ The idea "see whether the different languages play a role in the scores on the exams" sounds a bit imprecise like this is an exploratory data analysis. Have you thought about how you will interpret the coefficients in the models you are considering and what they tell you about your study? $\endgroup$
    – dipetkov
    Aug 28, 2022 at 15:10
  • $\begingroup$ So, I'm using dummy coding for Language. Hence, Spanish is the reference level (0) and English is (1). I've ran a simple regression with lm() for an illustration, Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 147.712 4.924 30.000 < 2e-16 *** LANGUAGEL2 -35.076 7.089 -4.948 2.66e-06 *** So my interpretation is "the score decreases by 35 points when the task is performed in Eng in relation to when its performed in Span" (b0 = 147, b1 = -35) $\endgroup$
    – Larissa Cury
    Aug 28, 2022 at 16:05

1 Answer 1

Reset to default
1
$\begingroup$

Proficiency is a continuous variable, ranging from 0 to 100. It's not clear how you propose to include it as a random effect. Unless you bin it into levels of proficiency but why would you want to do that? See the discussion on Why should binning be avoided at all costs?

Instead treat proficiency as a covariate, that is, include it in the mean structure of the model: ExamScore ~ Language + Proficiency. Optionally, you can "spline" it to allow for a smooth, flexible and nonlinear relationship between proficiency and test scores. However, before including Proficiency in the model at all, consider carefully what it measures: English proficiency (as a second language) before the test? after the test? throughout the school year? It might not make sense to have this variable in the model at all.

How do I enter a continuous variable as a random effect in a linear mixed effects model? explains that a random effect is conceptualized as a population of groups (students) from which we have drawn a random sample (students who have taken English and Spanish language tests).

Improve this answer
$\endgroup$
6
  • $\begingroup$ thank you! I like this ExamScore ~ Language + English_Proficiency idea but I have two data points for Language (ie, one score in Eng and one in Span) and only one score for Proficiency (therefore, each subject has its Eng proficiency duplicated in the table), that's why I was thinking about a random intercept (thinking that each subject will start from a different point considering their Eng scores) what do you think about it? $\endgroup$
    – Larissa Cury
    Aug 28, 2022 at 14:56
  • $\begingroup$ Think about what the formula ExamScore ~ Language + English_Proficiency + (1|Student) says for the expected score of student with proficiency 95 and for student with proficiency 100. $\endgroup$
    – dipetkov
    Aug 28, 2022 at 15:09
  • $\begingroup$ thank you for this link, it was very useful! I've ran the model you've asked me about Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) 118.4777 7.6677 80.2972 15.452 < 2e-16 *** LANGUAGEL2 -38.2114 5.5403 53.7005 -6.897 6.21e-09 *** PROFICIENCY 0.4954 0.1068 65.8902 4.639 1.71e-05 *** I'm confused, though, as to what extend does it make sense to use Proeficiency as a predictor? For ex, b0 = Score for Language when Lang = Spanish, but I don't have a proficiency score for Span (it seems weird to compare a measure for Eng to Span) $\endgroup$
    – Larissa Cury
    Aug 28, 2022 at 16:25
  • $\begingroup$ It's up to you to decide what makes sense for the analysis or not. I'm just pointing out a technical difficulty with the proposed analysis. You have a dataset, what you don't seem to have is a clear idea of what you want to ask of your data. I suggest you think about this more. PS: Not all questions we might want to ask can be answered by the data. $\endgroup$
    – dipetkov
    Aug 28, 2022 at 16:32
  • $\begingroup$ Some things to consider: What is the proficiency supposed to measure? What about the test scores? How are they plausibly related? What's the purpose of building a model for these test scores? ... $\endgroup$
    – dipetkov
    Aug 28, 2022 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.