0
$\begingroup$

I am doing analysis on an educational product that aims to predict what impacts whether or not a student gets a question correct or incorrect. The DV includes item scores from four different question types. For two question types, True/False and Multiple choice, the value per item is dichotomous, i.e 0 or 1. However, for the other question types, the value returned for each item ranges from 0 to 1, i.e. the value is continuous. My understanding is that psychometricians refer to these types of items as polytomous. My question is whether all of these question types can be included in a single predictive model, or whether this violates certain assumptions. Also what type of model would you recommend? Linear regression, logistic regression with weighting, decision tree, something else?

For more context, some of the independent variables are length of the question, number of choices, subject, grade level, etc. (Edit starts here) For question types that had a score of 0 or 1, then a logistic regression would make sense. For the other "continuous" type, a linear regression may make sense. I would prefer to run a single predictive model to facilitate comparisons in my independent variables. For example, I'd like to know if math content is more difficult than English content. I would also control for question type. A proposed model might look something like:

score ~ question_type + subject + grade_level + item_seen_before

Regarding the levels of the controls and IVs, they look like:

  • question_type: 4 levels
  • subject: 2 levels--Math and English
  • grade_level: 3 levels--elementary, middle, high school
  • item_seen_before: Dichotomous--yes/no

Thanks in advance for any assistance.

$\endgroup$
7
  • $\begingroup$ The student either gets the question right or wrong, correct? $\endgroup$
    – Dave
    Sep 21 at 12:54
  • $\begingroup$ For dichotomous questions, the value is 0 or 1, i.e. incorrect or correct. But for a multiple select question, for example, the score for a single item could be 0, .4, .6, .2, 1.0, etc. In other words, the range in scores for these types of questions can be from 0 to 1. If I just had question types that had a score of 0 or 1, then a logistic regression would make sense. For the other type, a linear regression would make sense. But I'd like to combine the two types into a single model...if possible. $\endgroup$
    – Mezy
    Sep 21 at 13:06
  • $\begingroup$ Why do you want a multivariate model instead of separate univariate models? Also, for your "continuous" items, are only those scores available, or could they take any value between 0 and 1? If the former, ordinal regression may be a good univariate models for those variables; if the latter, ordinal beta regression would be. $\endgroup$
    – Noah
    Sep 21 at 14:38
  • $\begingroup$ @Noah Having a single model would facilitate comparison between the IVs of interest. The "continuous" variables don't have an infinite range, but rather depend on the number of choices. Each choice is treated as a T/F question within a multiple select item, so an item with 4 choices will have different scores than an item with 8 or 9 choices. But the range will always be 0 to 1. I am not familiar with ordinal beta regression. Does that still sound like a viable option? $\endgroup$
    – Mezy
    Sep 21 at 15:04
  • $\begingroup$ Can you explain (in an edit to your question) what you mean by "facilitate comparison between the IVs of interest"? So it sounds like your "continuous" items are actually discrete, and could perhaps be analyzed using ordinal regression or separate binary regressions for each T/F sub-item. Ordinal beta regression would not be a good fit. $\endgroup$
    – Noah
    Sep 21 at 15:15

1 Answer 1

0
$\begingroup$

In most cases regression models only include a SINGLE dependent/outcome variable. You can include any number of independent variables (binary or continuous) but each model can only "predict" a single thing. All of that is true regardless of what type of model you are using. The different "flavors" of models are designed to analyze different types of dependent variables: you use a logit if the dependent variable is binary, an OLS if it is continuous and more or less normally distributed, a tobit model if it is continuous but censored etc.

There are "multivariate" regression models that include more than one dependent variable (an example in Stata here). However, these methods may be more complicated than what it sounds like you have in mind - especially if the different outcomes variables are not all on the same scale.

The simplest option would just be to run a separate model on each outcome, and choose the right model for the job each time. For likert scale items (e.g. "not at all, a little, somewhat, very much") use an ordered logit model and for binary ones use a binary logit model. It might be that different things predict different outcomes.

An alternative would be to try and COMBINE a set of outcomes into a single dependent variable, which could then be predicted using a single regression model. But if you want to do this you really want to first confirm that all of the observed variables are all measuring the same underlying construct, like "intelligence" (this could be done via factor analysis or PCA). Once you have done that then you could combine the variables in some way (either with a simple average, or by generating factor scores from the factor analysis, or some other method) to create a new (continuous) variables that could then be used as the dependent variable in an OLS model. However, if some of those variables are binary and others are continuous then you can't just compare them in a factor analysis or average them together, because they are on different scales. One solution to that problem might to be to take the z-score of all the items before averaging them together. A simpler approach would be to dichotomize the ordinal items first, but this involves throwing away useful data. There are various complications involved here...

$\endgroup$
1
  • $\begingroup$ The "multivariate" example from Stata that you provide is a multiple-outcome model, not just multiple-predictor. The word "multivariate" is best reserved for multiple-outcome models, with things like "multiple regression" for having multiple predictors. Thus the example is a "multivariate multiple regression." All outcomes in the example, however, are continuous. Multivariate OLS modeling gives the same point estimates as separate models, but adjusts coefficient covariances to account for correlated outcomes. $\endgroup$
    – EdM
    Sep 21 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.