Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a study where many outcomes are represented like percentages and I'm using multiple linear regressions to asses the effect of some categorical variables on thes outcomes.

I was wondering, since a linear regression assume that the outcome is a continuous distribution, are there methodological problems in applying such model to percentages, which are limited between 0 and 100?

share|improve this question
1  
Are these percentages continuous (like the percentage of cream in milk, for example), or discrete (like binomial proportions, a count in some category out of a total count)? –  Glen_b Jun 17 at 23:24
    
Uhm... i don't get the difference. Aren't they both continuous? Anyway I think the second describe better my data, since we are speaking about people out of at total. –  Bakaburg Jun 18 at 14:50
    
The distribution of counts divided by counts is definitely discrete. Indeed, the numerator is usually modelled as a binomial, the denominator is conditioned on (treated as constant), so the ratio is usually treated as a scaled binomial. However, even if the denominator was alse a random variable, the ratio would still be discrete since its sample space is countable –  Glen_b Jun 18 at 21:28
add comment

3 Answers 3

I'll address the issues relevant to either discrete or continuous possibility:

  1. A problem with the description of the mean

    You have a bounded response. But the model you're fitting isn't bounded, and so can blast right through the bound; your fitted values may be impossible, and predicted values eventually must be.

    The true relationship must eventually become flatter than it is at the middle as it approaches the bounds (or reaches it, if the function is continuous but not smooth), so it would be expected to bend in some fashion.

  2. A problem with the description of the variance

    As the mean approaches the bound, the variance will tend to decrease as well, other things being equal. There's less room between the mean and the bound, so the overall variability tends to reduce (otherwise the mean would tend to be pulled away from the bound by points being on average further away on the side not close to the bound.

A model that deals with such a bound should take such effects into consideration.

If the proportion is for a count variable, a common model for the distribution of the proportion is a binomial GLM. There are several options for the form of the relationship of the mean proportion and the predictors, but the most common one would be a logistic GLM (several other choices are in common use).

If the proportion is a continuous one (like the percentage of cream in milk), there are a number of options. Beta regression seems to be one fairly common choice. Again, it might use a logistic relationship between the mean and the predictors, or it might use some other functional form.

share|improve this answer
add comment

It might be worth investigating beta regression (for which I understand there is an R package), which seems well suited to such problems.

http://www.jstatsoft.org/v34/i02/paper

share|improve this answer
3  
You answer would be even better if you hit on some of the major reasons why linear regression suffers when the outcome is a percentage. –  Alexis Jun 17 at 18:36
add comment

This is exactly the same thing as the case when the outcome is between 0 and 1, and that case is typically handled with a generalized linear model (GLM) like logistic regression. There are lots of excellent primers for logistic regression (and other GLMs) on the internet, and there is also a well-known book by Agresti on the topic.

Beta regression is a viable but more complicated alternative. Chances are logistic regression would work fine for your application, and would typically be easier to implement with most statistical software.

Why not use ordinary least squares regression? Actually people do, sometimes under the name "linear probability model" (LPM). The most obvious reason why LPMs are "bad" is that there's no easy way to constrain the outcome to lie within a certain range, and you can get predictions above 1 (or 100% or any other finite upper bound) and below 0 (or some other lower bound). For the same reason, predictions near the upper bound tend to be systematically too high, and predictions near the lower bound tend to be too low. The math underlying linear regression explicitly assumes that tendencies like this don't exist. There typically isn't a great reason to fit an LPM over logistic regression.

As an aside, it turns out that all OLS regression models, including LPMs, can be defined as a special kind of GLM, and in this context LPMs are related to logistic regression.

share|improve this answer
1  
Although overall much of this answer looks worthwhile, it contains some misinformation that could confuse readers. The account of logistic regression in the first paragraph sounds like a description of a log-like transformation of the dependent variable followed by linear regression: it is not logistic regression. The interpretation of the coefficients is not quite right, either. A more important problem with "LPMs" is that when data are near the extremes, they likely exhibit asymmetric distributions of the residuals, which is an important violation of the iid assumption of regression. –  whuber Jun 17 at 21:50
    
I didn't think it was worth getting into odds ratios and such. I'll just strip that stuff out and let the OP read up on it then. Also good point about the residuals. –  ssdecontrol Jun 17 at 22:59
    
(+1) Thank you for your constructive responses! –  whuber Jun 18 at 0:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.