1
$\begingroup$

Let's say I have data on how much time (in hours) students spend doing homework per day at various schools across the United States. The values I have collected include: the average time spent on homework (aggregated at the school level), the average and maximum values for time spent on homework for each decile bin, and the standard deviations for both of the previous categories.

I have a hypothesis that the distribution of homework hours is a key predictor of some school-level outcome (let's say the school's rating). Specifically, I believe (counter-intuitively) that schools for which only a handful of students are working extremely long hours on homework are likely to do better than schools which have more symmetrical and even distributions of homework time.

So imagine this incorrect specification:

lm(school_rating ~ average_homework_hours + sd_homework_hours)

I'm concerned that:

  1. You can't have sd_homework_hours in the regression, as it a function of the mean (which you can't therefore hold constant).

  2. The coefficient on sd_homework_hours doesn't actually capture the hypothesis of interest (it's more a measure of dispersion than the idea of the shape of the distribution). Perhaps what I'm trying to get at is something like skewness, or a Gini coefficient?

So I'm wondering what the correct approach is to including some measure of how asymmetrical the overall homework burden is. Are there other metrics that better capture the idea of the majority of a distribution being centered on a few individuals? Can you use skewness as an IV measure in a regression specification? Is this the right approach?

Edit:

Just dawned on me that there's another way to think about this. Imagine lining up all the individuals who have completed a test at a given school and you rank-order them from least amount of homework time to most amount of homework time. You then plot a cumulative density plot of the total amount of homework time.

You can imagine schools where the addition of each student adds a fixed amount per student (so a perfectly increasing line). There are other schools where each additional kid barely adds anything to the total, until you get to the kids at the upper-end of the decile, at which point you see sharp increases.

What measure allows me to compare "sharp, peaked" increases in the plot versus slow and gradual increases?

Edit: Bumping this to promote visibility.

$\endgroup$
2
+50
$\begingroup$

You have the deciles of you distribution of interest, don't you? Use those!

Of course you may compute any kind of strange measure on them, even an approximation of skewness using means of decile bins, or deciles bonduaries themselves, but I would rather make it simple:

lm(school_rating ~ average_homework_hours + last_decile_mean)

I suppose that average time spent for homework by all students is a valuable predictor, so that's why I used it. However, if you think that nerdest students play such an important role, that should be captured by the other predictor, which only consideres the average time spent for homework by those 10% students who commit the most.

Edit: about collinearity

In the comments the point was rised that the two variables above are very correlated. Actually this is not that much of a problem, because it can be easily solved. Supposing your goal is to make inference on the parameters and test specifically for the second one being different from zero, you have many options:

Do nothing: you may just estimate the model as described above, collinearity will not prevent the model from being estimated correctly unless it is very extreme (correlation between them numerically no different from 100%), but variances of the estimated parameters will be large (and the correlation between estimators will be too important to be ignored). In this case usual t-test is not reliable, instead you can lean on CANOVA/LR test that compare performance of the two variables model with the one with just the main variable average_homework_hours. You can trust those test even with highly correlated variables.

Orthogonalize the second variable with respect to first one. This way the variances of the two estimators will be minimized, because their covariance will be 0. It is worth noting that the resulting model will be actually no different, this is just a way to estmate first parameter on its own, while reducing its estimated variance (you are practically subordinating the estimation of the second parameter to the first one) and also make t-test viable for the second parameter. If you care about interpretability, pick one other option, because this one makes it very harder (actually it is possible to make it easy again, but then you have to explain the method, and its ratio is not so easy to understand, so it is the same in the end).

Reduce collinearity, while keeping the model interpretable. The best option here seems to be to create a new variable gap = last_decile_mean - average_homework_hours, and use that one instead of simply the averaged measure for the last decile. This variable is easy to explain (the mean difference in homework hours between the 10% of the students who study more and the average student) and it will likely be less correlated than last_decile_mean. Still, it may likely be someway correlated, so it is more sound to use a CANOVA/LR test between two models as described above, instead of a simple t-test. One benefit of this option is that the estimated parameters will have lesser variance than the ones estimated without editing the variables.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The problem is that the tenth decile is the one that picks up the most extreme outlier -- e.g. the max value of the whole distribution. This makes it not particularly useful, as the entire tenth decile could be biased by an extreme outlier. If we look at the 8th or 9th decile, these values are correlated with average_homework_hours at around .95, so I'm concerned extreme multicollinearity will make the coefficients uninterpretable. $\endgroup$ – Parseltongue Nov 4 '19 at 14:42
  • $\begingroup$ I was meaning average time spent on homework for last decile bin with the variable last_decile_mean, not the absolute maximum value. You may as well average it with the mean of the previous quantile, if you feel that the proportion of students influencing the result should be 20% rather than 10%. In any case, collinearity is an easily addressable problem, for instance you may orthogonalize the second variable with respect to the main one (the ratio is that you won't use it without the first, the same thing is often done with polynomials) $\endgroup$ – carlo Nov 4 '19 at 17:04
  • $\begingroup$ Right, the average of the last_decile is also highly collinear with average_homework_hours (.93). Can you flesh out a bit about how you would specify an orthogonal variable with respect to the first? We are worried about interpretation, and have had problems in the past with orthogonal polynomials and interpretation. $\endgroup$ – Parseltongue Nov 5 '19 at 21:07
  • $\begingroup$ I just edited the answer to address these worries $\endgroup$ – carlo Nov 6 '19 at 14:55
  • 1
    $\begingroup$ you can see how to do it here: cran.rstudio.com/web/packages/matlib/vignettes/gramreg.html. easier way is to substitute the second variable with the residuals of the model regressing it, while using the first variable to predict it. $\endgroup$ – carlo Nov 6 '19 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.