3
$\begingroup$

I'm considering some projects in the future which require modeling literacy outcomes as composites (such as word reading scores), which will naturally never have negative values (it's impossible to read negative words on a page), and is capped by the maximum score one can attain (for example somebody reading $50/50$ words). Because of this, fitting a Gaussian model doesn't seem accurate because the data for a composite will normally be something like $0 \le x \le \text{max}_x$. However, I am not as experienced in this issue and have only used bounded data in one analysis so far, which used binary outcomes. This is quite straightforward to model with logistic regression because it is simply bounded between the two values $0$ and $1$ and can be fitted with the typical link function $g^{-1}(u) = \text{logit}^{-1}(u)$. This is however not quite the same as what I need.

Some alternatives that I have seen that look potentially more realistic:

  • Poisson regression: typically used for counts anyway. As I have read, it uses the link $g(u) = \text{exp}(u)$ to achieve some of the same things logistic regression does. But what I normally see with it's application is right skewed data, such as the Prussian army horse data. I've seen plenty of composites used in my area that follow a more normal distribution, and so I feel that fitting Poisson regressions to this outcome data would yield overdispersion, as the variation may be greater than expected in the model. So far, I am undecided on if this method should be chosen.
  • Logistic regression: seems to achieve the same thing by modeling proportions of successes rather than just the outcomes with each trial. This would make sense for a composite measure as they are technically proportions anyway (successes/trials). If simply fitting the data as a logistic regression the typical way works for this, then it immediately solves some of my problems, as I already have some experience with this approach. This so far seems to be the best option.
  • Probit regression: appears to model the data with a "normal distribution", where $\text{Pr}(y_i = 1) = \Phi(X_i \beta) $. However, what this actually means in practice to me is lost because I keep reading that it's usually only appropriate for binary outcomes. I have also read that for many purposes, logit/probit can be treated pretty similarly. This is my least favorite option because functionally it doesn't seem to add much and may confuse reviewers for no reason.

Typically I never see this issue considered at all in my field, so I don't even know if it's worth any serious thought. However, given what reading I have done on this subject, treating this kind of data with a Gaussian-distributed error model seems either limited or hazardous.

Edit

I tried simulating this kind of data to see if it was reasonable. Below is some R code that I used:

#### Simulate Data ####
library(tidyverse)
set.seed(123)
n <- 100
b <- 1
total <- 30
prob <- .5
x <- rbinom(n,total,prob)
x <- x-mean(x)
y <- ( (b*x) + rbinom(n,total,prob) ) 
dv <- y/total
df <- data.frame(x,dv)
cor(df)

#### Fit Data ####
fit <- glm(dv ~ x, 
           family = quasibinomial)
summary(fit)
exp(coef(fit))

#### Plot ####
df %>% 
  ggplot(aes(x=x,y=dv))+
  geom_point()+
  stat_smooth(
    method = "glm",
    method.args = list(family=quasibinomial),
    formula = y ~ x
  )+
  labs(x="X",
       y="Y",
       title="Simulated Quasibinomial Regression")+
  theme_bw()

Which shows the fit seems reasonable:

enter image description here

But when I check the fitness measures, some of them look poor when I run plot(fit):

enter image description here

$\endgroup$
2
  • $\begingroup$ Censored regression. stats.stackexchange.com/questions/49443/… $\endgroup$
    – rolando2
    Oct 1, 2023 at 0:47
  • $\begingroup$ I will say in practice that censoring itself isn't the bulk of the problem for what I was considering (generally people score within a proportion of 0 and 1), but it does come up and this is an interesting method to address that. $\endgroup$ Oct 1, 2023 at 1:01

1 Answer 1

1
$\begingroup$

Some thoughts: First, probit and logistic are alternatives to each other that (usually) yield similar results. The biggest difference seems to be that they are typically used in different fields. Psychology tends to use logistic. I believe economics uses probit more often. There are many, many places on the 'net that compare and contrast the two, but, essentially, if one is appropriate, the other is too. Neither one is more "confusing" than the other, it's more a question of what you are used to.

Second, you do have a count, so count models have their appeal. However, Poisson regression makes assumptions that are, in my experience, almost never met, so, usually, an alternative such as negative binomial regression winds up being needed. The nice thing about these models is that they are for counts. But they aren't bounded. So, your model could predict more than 50 correct, which doesn't make much sense. If you are interested in the proportion correct, logistic might be better.

Third, if the number correct covers a fairly wide range and there aren't many people with very few correct, then "regular" OLS regression may not be bad. The violations of the assumptions may not be bad. You can check.

$\endgroup$
2
  • $\begingroup$ Also see stats.stackexchange.com/a/627637/4253 $\endgroup$ Sep 30, 2023 at 13:41
  • 1
    $\begingroup$ +1. Just for clarification, the part about confusion is more about educating an audience about what exactly it does. Sentiments like "we don't do that in psychology" are sometimes a barrier in peer review I would like to avoid if there are no clear advantages to probit over logit. I also worry about using logit because I don't see people using this for anything but binary data, but it would be more familiar at least to a psych audience. Would there ever be a case where Gaussian regressions would fit the data better for proportional data, or would binomial regression always do a better job? $\endgroup$ Sep 30, 2023 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.