Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a set of data where the response is a proportion. For each event in the experiment, a system will correctly tag X of Y items. In the end, I will have something like 100 different results. I have been trying to determine what is the most correct method of fitting the response variable. So far I have uncovered 4 number of options.

  1. R's built-in glm using one line for each of the Y items in each experimental event
  2. R's built-in glm using a proportion (X/Y) with the weight set to Y for each experimental event
  3. The betareg fit method from the R betareg package
  4. Quasi-likliehood method proposed by Papke and Wooldridge (pdf)

I am unsure how to rank these different methods to know which is the most appropriate. Here are some specific questions:

  1. How does the likelihood change for these different options (especially between options 1 and 2)?
  2. What are the pros and cons of the different methods?
  3. Do these methods have limitations on the types of factors that can be built into the model?
  4. For the betareg function, are the weights specified the same way as the glm function?
  5. Most people would tell me to use option 2 listed above, but I have seen a lot of pull for option 3. Why should I choose option 3 over option 2?
  6. What do you do if you do not know the denominator of the proportion (i.e., you cannot properly weight the data for option 2)?

As you can see these questions are quite general and could be the elements of a full academic text. I was just not sure where to go to get the answers to these questions.

share|improve this question

2 Answers 2

up vote 1 down vote accepted

The model's reported degrees of freedom in the output will differ between 1 & 2, but this won't have any effect on anything else. For comparisons between models, so long as both were fit grouped or both ungrouped, there will be no difference. If you know the number of trials & the number of successes, your data are binomial, so you use logistic regression as @Glen_b notes (you can flip a coin between 1 & 2).

If you know the number of successes, but not the number of trials, you can fit a count regression, (probably a negative binomial or other over-dispersed Poison).

If you have a proportion, but don't know either the total number of trials or the number of successes, you will have to fit a beta regression. Beta regression is for continuous proportions, so it wouldn't be ideal, but if the proportions were based on a large number of trials, it probably wouldn't be too biased.

share|improve this answer
    
With regard to your comment on the Beta regression... You say that Beta regression if for continuous probabilities. Some example data sets are something like fraction of income spent on food. Here, the fraction could take on any value on the interval [0,1], so I would say it was continuous. However, I could also lay each dollar made and then make a binary decision for each dollar effectively giving me a proportion with a specific number of successes and failures. What is the difference in these two procedures and how it would affect which analysis to use? –  Justace Clutter Apr 22 at 11:31
    
A continuous proportion can take any value on the interval $(0,1)$, whereas a discrete proportion has 'gaps'--values w/i that interval that can never be observed. Eg, on 4 coin flips, you can get 0, .25, .5, .75, or 1, but none of the values in between. You have Y discrete items, of which X are tagged correctly, that means the only proportions possible are from 0 to 1 in increments of 1/Y. Note also, the binomial can have 0 & 1, but the beta can't. –  gung Apr 22 at 13:58

You're making it too complicated. Don't model the proportion, model the success/failures. Your data come from a Binomial distribution (X successes, Y trials). Therefore use a logistic regression model.

glm(formula = cbind(X, Y) ~ ..., family = binomial)
share|improve this answer
    
Thanks for the feedback. One issue related to this is the fact that one tag may improve the probability to successfully tag the next one. Therefore, the individual object tags are correlated. I was under the assumption that in order to use the Logistic Regression in this way the individual trials should be independent. –  Justace Clutter Apr 22 at 11:26
    
You can examine the residuals to assess the independence assumption and other potential violations. –  Glen Apr 22 at 16:21
    
You suggest to investigate the residuals. In this case that would be the likelihood residuals. For a OLS fit I understand that the residuals should be uniformly distributed and exhibit a homoskedastic distribution. I am not sure what I am looking for in the likelihood residuals for a logistic regression. (I will go ahead and hit up a text on this but a few comments from you might go a long way) –  Justace Clutter Apr 22 at 17:04
    
It's similar to OLS, plotting the deviance residuals by sample and looking for any type of pattern would violate independence. r-bloggers.com/… –  Glen Apr 22 at 18:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.