Multiple nominal categories predicting a continuous variable

Question

This is a simplified version of the problem that I'm having, but I'm really looking for the appropriate type of analysis for the question in which I'm interested.

Set up: I have a large data set in which students provided a rating (from 1 to 7) on the quality of the "faculty" in a course. There were multiple different courses with multiple different professors and multiple different teaching assistants. The same professor could teach multiple courses, and the same teaching assistants could assist in different courses for any professor. Note that students did NOT repeat across courses.

Therefore, there are four variables:

• course rating (1 to 7) from a survey
• course name
• professor name
• TA name

For the moment, let's pretend that there are three courses (A, B, C), three professors (R, S, T) and three TAs (X, Y, Z). Therefore the dataset would look like this, with each row being a response from an individual student:

A, R, X, 4
A, R, X, 5
A, S, Y, 7
B, T, Z, 3
etc.

As I said, there are a lot of rows here (and far more than three courses, three profs, and three TAs).

The question I'm interested in is this:

What is the best way to determine which predictor variable (course, prof, TA) is most influential in determining course rating?

Within a category (course, prof, TA), I do NOT care if A is better than B, or X is equal to Y, etc. -- I'm interested in which category is most influential.

Thank you in advance.

Answer 1

This is not an easy problem, as disentangling it can come to face several issues. Your dependent variable (the thing you want to predict) is the score (which is not continuous as states, but a discrete interval), and you have three explanatory variables that are non-ordered categorical - class, professor and TA.

Logically, you can expect that the only way to be able to decide which is more important is if we have enough data with enough partial repetition. That means that course c_a is taught by several professors p_a, p_b, and p_c, each with several TAs t_a, t_b etc.. For instance, if c_a with three professors and the same TA t_a get the same score, you can say that the professor does not matter. But which is more important - the course or the TA? If it's the course, than the same course taught by other TAs would get similar scores, and if it's the TA, than the same course with different TA's will get different scores. But the plot thickens - often TAs work with the same professor in several courses, so the error terms will not be independent for them. Moreover, even the same professor (as an example) can be better or worse depending on the course - it is common for professors to teach courses that align with their interests and also be forced to teach courses they do not want to teach (especially young professors). So you can imagine an interaction between specific professors (as well as TAs) and courses, and also between TAs and professors (e.g., t_b works best with p_c, no matter the course).

Now, since there are (I assume based on your description) many many courses, professors and TA's, interacting them will become very VERY unwieldly as a model with a lot of degrees of freedom taken.

So, you can try to run a regression model (perhaps consider Poisson count regression). Pay special attention to the model assumption - doubly so to the independence of errors. Create interactions, and see the results.

However, the above will not solve your problem! That is just the best you can do with your data as described. It might tell you that specific professors, or specific TAs are more associated with higher or lower course scores. It will not tell you if professors in general or TA's in general are more influential - you don't have courses that have no TAs or that are only taught by TAs, in which case you will be hard pressed to say anything of value on professors or TAs as a whole.

You can, I guess, look at relative importance of the beta scores of specific professors and TA's and aggregate them and compare to get an indication... I have never done anything like this.