This is an extension of Goodness of fit for a logit-transformed linear random-effects model?. Here is the setting of the problem, inserted below for convenience.
There is an examination that students (indexed by $i$) can take once annually in a three-year program. Students are not obligated to take this examination each year, but are required to take it at least once over the three years.
A student could (but is not obligated to) take an exam-prep class of sorts either before or after an exam at time $t$ (indicated by $\alpha_{it}$) and may be in years 1, 2, or 3 of the program (indicated by $\beta_{it}$). For our purposes, if student $i$ never takes the prep class, $\alpha_{it} = 0$ for all $t$. The value $t$ indicates the calendar year in which the exam was taken, distinct from the year the student is in the program.
This was the original way I had considered modeling the problem:
Suppose I have a linear mixed-effects model $$\log\left(\dfrac{y_{it}}{1 - y_{it}} \right) = \mu + b_1\alpha_{it} + b_2\beta_{it} + \gamma_i+\epsilon_{it}$$
where $y_{it} \in (0, 1)$ is the exam score expressed as a percentage of the $i$th student at time $t$(truncated from above at 0.999), $\mu$ is an intercept, $\alpha_{it} \in \{0, 1\}$ (a binary indicator equalling $1$ if an exam-prep class was taken before taking the exam at time $t$), $\beta_{it} \in \{1, 2, 3\}$, $\gamma_i \sim \mathcal{N}(0, \sigma^2_{\gamma})$ is a random effect used to incorporate student-to-student variability, and $\epsilon_{it} \sim \mathcal{N}(0, \sigma^2_{\epsilon})$ is the usual noise term. $b_1$ and $b_2$ are your usual coefficients estimated through least squares (taking into account the random effect).
Based on one of the comments I received in the original question, one suggestion was to use a Gamma generalized linear mixed-effects model instead, so something like $$\log(y_{it}) = \mu + b_1\alpha_{it} + b_2\beta_{it} + \gamma_i$$ where we assume $y_{it}$ is Gamma distributed. This is great for me, because I can use deviance testing to compare models, but this makes me wonder the following:
- How do I know that a Gamma assumption for $y_{it}$ is reasonable, other than that $y_{it}$ must be a positive value?
- Why would I choose a log link over a different link?
- How would I know that $\gamma_i$ being normally distributed is a reasonable assumption?
Journal articles and textbooks are appreicated.
