1
$\begingroup$

I am trying to predict first term GPA for college students based on a number of incoming factors (high school gpa, placement test, year). This isn't the overall model just a simpler one. The first term GPAs are on the interval 0 to 4, however the predictions from the linear OLS model (i'm using sklearn) never go above 3.6 (see picture). Is this some sort of gotcha that I am missing? There are certainly data in the training set with first term GPA that is between 3.6 and 4.0. I didn't expect perfect performance but this is odd to me. enter image description here

enter image description here

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Can you show us the model diagnostic plots? $\endgroup$
    – user2974951
    Commented Dec 21, 2018 at 10:26
  • $\begingroup$ i added model diagnostic plots. i think the cooks distance is wrong though because i had to calculate it myself because i didn't find any function outside of statsmodels that implements it and i havent written tests or compared it to output from R, statsmodels, etc. $\endgroup$
    – mnky9800n
    Commented Dec 21, 2018 at 13:24
  • $\begingroup$ Just by looking at the first (diagnostic) plot you can see that your model is fu****. Your dependent variable is limited to 0 and 4 so a regular linear model is not valid. Look into beta regression. Also, do you have repeated measures in your data? As in, do some students have more than 1 score? Is there any kind of correlation going on? $\endgroup$
    – user2974951
    Commented Dec 21, 2018 at 13:30
  • $\begingroup$ yeah this is my thinking as well and will check out beta regression. students should only have a single score although some are averages (e.g., the output variable is the first semester GPA which is the average of several scores). $\endgroup$
    – mnky9800n
    Commented Dec 21, 2018 at 15:48

1 Answer 1

Reset to default
3
$\begingroup$

Predictions like these don't include the 'error' in your model: that is, you expect that even if your model is very good, a student with some combination of predictors will not be exactly the prediction, they will be above or below it. The only way you would get a prediction of 4 would be if a combination of predictors gave an estimate that the average GPA for that observed combination would be 4. If the average GPA for that observed combination is 3.6 with a range of 3.0-4.0, your model would predict 3.6 even if you would expect some fraction to have 4.0; your best guess for each individual student is the expected value, the mean. If you instead imagined your predictions as probability distributions you would find that these include 4.0.

However, it looks like your model itself is not very good: there is a lot of difference between your predictions and outcomes and very little slope between them relative to the variance.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yeah I agree that the model is not very good. I guess I got stuck on maximum prediction value, and wondering why it wouldn't be higher than trying to tune it into something that is predictive. $\endgroup$
    – mnky9800n
    Commented Dec 21, 2018 at 10:13
  • $\begingroup$ @Krause I very much need a formal citation for the argument you make clearly, about variance of predictions being lower than variance of obs. I have tried hard to find a paper(s) but so far without success. Suggestions please? Does the concept have a name on which I might search? Thank you. $\endgroup$
    – InColorado
    Commented Feb 5, 2021 at 5:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.