# Judging the quality of a statistical model for a percentage

I have a data set with multiple predictors, and a single response variable which is a percentage, thus is bounded between 0 and 100. I cannot share the dataset unfortunately. I would like to build a simple model for the response. I then tried, perhaps mindlessly, to use logistic regression using glm. R throws the following error:

    logistic_regression <- glm(y ~ x1 + x5 + x11 , data = df_red, family = binomial(link="logit"))
Error in eval(expr, envir, enclos) :
the values of y must be 0 <= y <= 1


The error message might not be exactly that because I had to translate it from my language (I had never seen R throw errors in my language before!). The concept is there, however, and it's right: my $y\in[0,100]$. I must rescale it so that $y^*\in[0,1]$. I then get

    logistic_regression <- glm(y/100 ~ x1 + x5 + x11 , data = df_red, family = binomial(link="logit"))
Warning message:
In eval(expr, envir, enclos) :
#successes not integers in the glm binomial model!


This time, I don't get an error but a warning. This makes sense: after all, the response variable for logistic regression should be a binary variable, not a continuous one. On the other hand, the model ran. My questions:

1. How do I judge the quality of this model? Does it make sense to look at residuals vs fitted, distribution of residuals, etc.? I am mainly interested in prediction, thus point estimates and prediction intervals for unseen data. A secondary goal would be interpretation of coefficients: if I increase $x_1$ by 1 ceteris paribus, does $y$ increase by a fixed amount? A fixed ratio? Neither of those? The third goal is inference on the coefficients: I care more about uncertainty estimates for $\hat{y}$, but if I can have confidence intervals for the coefficients of the model, that would be good, too.
2. Does the model make sense at all? Should I do something completely different, such as for example beta regression, or can I use something simpler/more similar to what I did?
• You don't say enough about what the response actually consists of ("percentage" doesn't even indicate whether its a ratio of counts or of continuous amounts) to give useful advice. – Glen_b Oct 31 '16 at 10:12
• @Glen_b, you are quite right, it didn't occur to me that a percentage could be something different from a ratio of counts. I cannot say what it is, actually, but you can think of it as "(# individuals who did A & B) / (# individuals who did B)". Does that help? PS of course "# individuals who did B" is not a constant: it depends on the predictors. I don't have access to the observations of "# individuals who did B": I guess I could retrieve them, but for various reasons I'd rather not do it unless it's strictly necessary. – DeltaIV Oct 31 '16 at 10:58
• Well those are count proportions - but is sounds like you don't have the denominators (which impact the relative variances of the observations). Do you have the numerators of the counts or only the ratios themselves? – Glen_b Oct 31 '16 at 11:42
• If I had the numerators, then I would have the denominators, since I have the ratios...give or take a little rounding error. – DeltaIV Oct 31 '16 at 12:10
• You're correct, but not everyone realizes - I've seen it happen before, a couple of times. – Glen_b Oct 31 '16 at 15:54

If your data (to be predicted) is pure fractions/percentages, then logistic regression is not really appropriate.

The closest continuous analogue would probably be to assume the data is logit-normal. I have done this before in similar situations and gotten reasonable results (however my "ad hoc" approach was not really statistically rigorous).

In the more statistically rigorous GLM framework, you could still use a logit link function as in logistic regression, but the (conditional) data distribution will no longer be Bernoulli distributed. As you hint in your last point, beta regression is probably the most common approach for predicting a fraction/percentage (i.e. you assume a Beta distribution rather than a logit-normal). This would certainly be more appropriate than logistic regression, and I imagine it should be straightforward in R (which I do not use, so YMMV).

The second factor to consider though, as mentioned in the comments, is the possibility of using the raw counts rather than the percentages. This will affect the relative uncertainty of your data, e.g. $0.5=\frac{1}{2}=\frac{500}{1000}$ but the second ratio has much less uncertainty than the first. So if you can use the raw counts that approach should be strongly preferred!

For integer count data, possible distributions would be Binomial or Poisson. For continuous "count" (i.e. non-negative weight) data, a Gamma distribution could be appropriate. (Note that the Beta distribution can be interpreted as the mixing-fraction for a binary mixture of two Gamma-distributed components.)

In the count case you can model the two "component-counts" as your primary variables, i.e. if $y=\frac{A}{A+B}$ then you can model $A[x]$ and $B[x]$. Or you could model $A$ and the "total mass" $C=A+B$, depending on the relative correlations. For example if $C$ is relatively constant, $B$ will be strongly correlated to $A$, so $C$ may be a better 2nd variable to target, i.e. more independent.

• thanks, great answer - couldn't read about logit-normal until now, but it seems very interesting. Also the practical advice on how to build a model when I have raw data is appreciated - for now I don't (see comments to my question), but it's interesting anyway. Just a question - in the GLM framework you say I could use a logit link function and a Beta conditional data distribution. I think you mean a conditional error distribution, right? In the usual regression framework we're not concerned with the distribution of $y$ and of the $x$, but with the distribution of the residuals. Right? – DeltaIV Dec 17 '16 at 12:06
• As I understand GLM, you assume a conditional data distribution $y\mid{x}$ and model its mean $\mathbb{E}[y\mid{x}]$ using a linear predictor transformed by the (inverse) link function $g^{-1}[Ax]$. In the Gaussian case you can think of the errors $y-\mathbb{E}[y\mid{x}]$ as stationary. But in general the conditional scale (e.g. standard deviation) of the error may depend on the conditional mean of the data. Not sure if this helps (and I may be wrong!) – GeoMatt22 Dec 17 '16 at 15:06