I'm currently analyzing some data to establish if school performance for particular subjects makes for good predictors of university performance at first year level across all faculties and degrees. In essence we want to know which specific school subjects are the best predictors of university performance so those can be used for admission purposes alongside admission tests. This has obviously been done a million times before.
The problem is that, due to recent curriculum overhauls, I only have access to one cohort's data for now. For various reason, some discussed below, regression the way I was taught might not be the way to go.
My first instinct was to do some hierarchical regression for each Faculty by entering the compulsory school subjects needed for admission (for example, math and science for engineering) first and then entering some other subjects of interest second (history and economics for engineering). The outcome variable would be 1st year GPA. This is probably a bad idea though for a number of reasons.
1) GPA, as a construct, will vary from degree to degree within faculties. For example, it would be very misleading to equate the GPA for a Human Resource Management degree and an Actuarial Science degree with one another, though they are both in the same faculty.
Theoretically this is a problem but practically as well. For example, despite very similar mean 1st year GPA scores, the mean school math and science mark for Actuarial Science is about 25% higher than the HR Management means. Regression won't like this very much. As far as I can see there's no real way to quantify how much more difficult certain clusters of university subjects are than other clusters making GPA sort of useless as an outcome variable at faculty level.
2) Ideally I'd like to do this at the level of individual degrees where there is some stability in terms of degree difficulty, but if I do this I end up chopping the student numbers severely. With all said and done I have around 5 degrees out of about a 100 yielding sufficient student numbers to run regression. I played around with some power calculations and with 5 or 6 predictors I'll need somewhere in the vicinity of 100+ students. However, due to the selectivity of some of these degrees, school marks tend to cluster at the higher end with very little variance between students and makes for less than ideal predictors. University GPA scores tend to be more dispersed though.
I'm hesitant to run regression on degrees with 80 students or less for example, especially given the real risk of collinearity issues with regards to school subjects. There is a pretty robust correlation between math and science marks in the data for instance and all degrees that require one also require the other for admission.
Any ideas? Help would be greatly appreciated as I'd hate to construct 100's of regressions with only a small number being usable.
