Logistic regression? multiple categorical and ordinal values to predict binary outcome

Question

I have two hypothetical examples I would love help with:

1) Let's assume you're analyzing the relationship between 10000 high school students' graduation outcome (Y/N; binary) with the letter grades (A-F; ordinal) from each subject (categorical label) they took. Not all students were required to take the same classes, so some students don't have letter grades for certain subjects.

Which model would determine if a high grade in one subject is a better predictor of graduation outcome (and which subject)? Based on my research over the last 3 hours... I was thinking a logistic regression, but I usually do all of this with a calculator and a ton of scratch paper, so I'm not following a ton of the online material regarding Python and r.

2) Building off of this example let's assume there are components of each subject, so in addition to your letter grade in the subject, you also have a letter grade for the component.

Let's take the subject of Math as an example. The components of Math that also receive a letter grade are:

• Algebra

• Lattice multiplication

• Data visualization

• PEMDAS understanding

This is across each subject, but not every component has a letter grade and not all the components are the same. Each subject has a varying number of components.

How could I determine if/which component was the largest predictor of graduation outcome?

Any help is appreciated because I am so tired of thinking about this on my own for hours!

Answer 1

From what I understand in your question you don't want to predict the graduation outcome but rather check which subject (your variables) grade has the most impact on graduation (As for your question regarding if high grade causes more graduation you can easily make a table or plot for that and check, As expected you'll see people getting higher grades have more Y). To check variable importance/impact there are many ways :

1. Model Specific Methods: These are the methods that check variable impact based on the model built. E.g :

• t-statistic for linear models like logistic regression you said
• In random forest there is an importance function
• Recursive Partitioning, it is the reduction in loss function for each of your variables
• Importance function in xgboost
• varImp function in Multivariate Adaptive Regression Splines(MARS)

And there are more.

2. Second category is model independent tests to check which variable has the most impact:

• The most basic test is to check correlation of the predictors (subject) with your response variable(Y/N). You can see check cor b/w categorical and numeric/categorical columns using the hetcor function in the polycor package in R

• Chi Squares Tests

• Information gain tests

You can google these methods to know more about them and you'll easily find tutorials on how to apply them in R/Python.

As for your second question to do component analysis, I suggest breaking down Math into different columns using one hot encoding(dummy), it will give you 0 if he has not taken the component and 1 if he has, and there will be corresponding graduation outcome for that.( You can find how to do it in R/Python easily on google). Then apply the tests that I said above for each subject individually.

For the first analysis(subject importance one) , I have a question, You said not all people have taken all the subjects, So some values will be missing, what is the amount of those missing values (percentage) in each column. You may have to do imputation if it is large, otherwise just delete them.

Answer 2

As your dependent variable (i.e., graduation) is binary (0/1) you should perform a logistic regression. This will measure the effect of your independent variables (e.g., student's grade, age, gender, etc.) on the probability to graduate. If your data are "structured" (e.g., the students belong to schools themselves nested within geo areas), then you should consider running hierarchical regression (e.g., mixed effects logistic).

In your case I am more concerned by the availability of the information for each student ("Not all students were required to take the same classes, so some students don't have letter grades for certain subjects"). This is an issue if you want to estimate the effect of "letter grade" on graduation, because in standard logistic regression this will be treated as missing values and your corresponding observation will be "automatically" removed from the data set. When you specify your independent variables, try to include only the ones which have a value for most of the students. For example, in your case you could first try to estimate a Logit model linking Proba{Graduation} with "type of classes" => For each subject and each student, you should be able to code a binary variable indicating whether the student took the subject.

Answer 3

Regarding your first question (only):

As I understand your question, your dependent variable is Graduation=1/0 and you want to assess how grades for different subjects (predictors) are related to the outcome (Y/N). However, as Umka noted, you may have issues with missing data because "Not all students were required to take the same classes, so some students don't have letter grades for certain subjects."

Depending on how many predictors you have, I would perhaps start with a model having the letter grade for each subject as a categorical predictor -- for instance, a series of dummies, leaving F as the reference category and using dummies for D, C, B, A plus "no grade for this subject". If you have grades for three subjects, you would have three sets of dummies like that.

There is certainly more sophisticated ways to tackle your problem but this would at least help to find what letter grade for a subject is associated with a significant increase in the probability of Graduation=Yes.