The Gamma distribution assumes that the outcome is positive, but it also allows that it can be greater than one, which according to your definition should not be allowable. Now, if the majority of the observed data are relatively close to zero, the Gamma model could still provide and relatively good fit.

An alternative is to use a model that respects the nature of your bounded outcome. One option is the logit-normal distribution that you used. But as mentioned in the comments in the original post, interpretation of the parameters can be problematic. Another option is to use a Beta mixed effects model. For this model the interpretation of the regression coefficients is easier because they directly relate to the mean of the distribution.

Also, to check the fit of the assumed distribution for your data, you can use the simulated residuals from the **DHARMa** package.

If you plan to fit the model in R, you can fit this model in the [**GLMMadaptive**][1] package. For an example, see [here][2]. And for an example checking the goodness-of-fit check [here][3].


  [1]: https://drizopoulos.github.io/GLMMadaptive/
  [2]: https://drizopoulos.github.io/GLMMadaptive/articles/Custom_Models.html#beta-mixed-effects-model
  [3]: https://drizopoulos.github.io/GLMMadaptive/articles/Goodness_of_Fit.html