Reasonableness of assumptions for a generalized linear mixed-effects model This is an extension of Goodness of fit for a logit-transformed linear random-effects model?. Here is the setting of the problem, inserted below for convenience.

There is an examination that students (indexed by $i$) can take once annually in a three-year program. Students are not obligated to take this examination each year, but are required to take it at least once over the three years.
A student could (but is not obligated to) take an exam-prep class of sorts either before or after an exam at time $t$ (indicated by $\alpha_{it}$) and may be in years 1, 2, or 3 of the program (indicated by $\beta_{it}$). For our purposes, if student $i$ never takes the prep class, $\alpha_{it} = 0$ for all $t$. The value $t$ indicates the calendar year in which the exam was taken, distinct from the year the student is in the program.

This was the original way I had considered modeling the problem:

Suppose I have a linear mixed-effects model
$$\log\left(\dfrac{y_{it}}{1 - y_{it}} \right) = \mu + b_1\alpha_{it} + b_2\beta_{it} + \gamma_i+\epsilon_{it}$$
where $y_{it} \in (0, 1)$ is the exam score expressed as a percentage of the $i$th student at time $t$(truncated from above at 0.999), $\mu$ is an intercept, $\alpha_{it} \in \{0, 1\}$ (a binary indicator equalling $1$ if an exam-prep class was taken before taking the exam at time $t$), $\beta_{it} \in \{1, 2, 3\}$, $\gamma_i \sim \mathcal{N}(0, \sigma^2_{\gamma})$ is a random effect used to incorporate student-to-student variability, and $\epsilon_{it} \sim \mathcal{N}(0, \sigma^2_{\epsilon})$ is the usual noise term. $b_1$ and $b_2$ are your usual coefficients estimated through least squares (taking into account the random effect).

Based on one of the comments I received in the original question, one suggestion was to use a Gamma generalized linear mixed-effects model instead, so something like
$$\log(y_{it}) = \mu + b_1\alpha_{it} + b_2\beta_{it} + \gamma_i$$
where we assume $y_{it}$ is Gamma distributed. This is great for me, because I can use deviance testing to compare models, but this makes me wonder the following:

*

*How do I know that a Gamma assumption for $y_{it}$ is reasonable, other than that $y_{it}$ must be a positive value?

*Why would I choose a log link over a different link?

*How would I know that $\gamma_i$ being normally distributed is a reasonable assumption?

Journal articles and textbooks are appreicated.
 A: The Gamma distribution assumes that the outcome is positive, but it also allows that it can be greater than one, which according to your definition should not be allowable. Now, if the majority of the observed data are relatively close to zero, the Gamma model could still provide a relatively good fit.
An alternative is to use a model that respects the nature of your bounded outcome. One option is the logit-normal distribution that you used. But as mentioned in the comments in the original post, interpretation of the parameters can be problematic. Another option is to use a Beta mixed effects model. For this model the interpretation of the regression coefficients is easier because they directly relate to the mean of the distribution.
Also, to check the fit of the assumed distribution for your data, you can use the simulated residuals from the DHARMa package.
If you plan to fit the model in R, you can use the GLMMadaptive package. For an example, see here. And for an example checking the goodness-of-fit check here.
