0
$\begingroup$

I want to fit an unordered categorical (multinomial) regression model in which I have two categorical predictor variables. It makes sense that these two variables interact with each other.

However, as effects in multinomial regression are hard to interpret on the log-scale or even the OR-scale. Also, interactions on the log-scale do not necessarily correspond to interactions on the probability scale. In fact, on the probability scale, all effects depend on one another, as outcome probabilities only exist given different configurations of possible predictors (or their levels) (see for instance Berry/DeMeritt/Esarey (2009)).

My question: Should I even bother with interactions for the logit effects when my plan was to inspect the outcome probabilities given different predictor configurations all along? Or do those probabilities change if I include an interaction effect on the log-scale?

$\endgroup$
1
$\begingroup$

The link to the Berry, et al. paper, their links to the King, et al. (2000) paper, this post on AndrewGelman.com (http://andrewgelman.com/2015/05/04/a-causal-inference-version-of-a-statistics-problem-if-you-fit-a-regression-model-with-interactions-and-the-underlying-process-has-an-interaction-your-coefficients-wont-be-directly-interpretable/), all point to one incontrovertible conclusion: this is a highly contentious topic about which there is little agreement in the literature. For instance, Berry, et al's "compression" effects in a main effects model sounds like little more than a lack of independence among the predictors to me. After Wooldridge, you may well be better off simply testing for endogeneity in the error term rather than jumping through all of Berry, et al's, convoluted hoops.

That said, one suggestion stands out for me in terms of identifying a line of sight through the controversies: testing unrestricted and restricted models for the change in the log-likelihood. If that test is significant, then you can plausibly contend that including an interaction term substantively improves the outcome.

Of course, this is a statistical answer to the question and, pace King, et al., the challenge then becomes one of providing substantive interpretations for this result, not statistical ones. If substantive interpretations can't be provided, then perhaps it makes more sense to drop these terms from the model.

You don't provide a context for your question: is this for a dissertation, a publishable paper or an applied model such as might be used in a corporation? This is another way of asking how high the bar is being set. My point here is twofold: first, the required level of rigor changes as a function of the context. If you're doing this for a dissertation then you can motivate your choices based on a plausible but careful reading and interpretation of the literature while recognizing that there are no unambiguous, black-and-white solutions. Second, slavish adherence to a contentious and convoluted literature will do little or nothing to advance the issues. Who knows? You may achieve a breakthrough in the academy's understanding if you push the envelope.

$\endgroup$
  • $\begingroup$ Mike... this was very quick and very helpful, thank you! I suspected that this was an unresolved issue and thought about doing as you suggested. I'll gather literature and present a few viewpoints and then ultimately will evaluate different fits and see if the models are interpretable with regard to subject matter. This is just for my master thesis. I'm a social scientist and have a categorical outcome variable that is the result of an unsupervised text classification via a mixture model. I want to predict text class membership according to categorical criteria (e.g. newspaper, subsection). $\endgroup$ – quadzar Jun 3 '15 at 12:31
  • $\begingroup$ Ugh! That sounds kind of messy. Don't be surprised if there isn't much in the way of fit to the "unsupervised text classification," unless your predictors were used in the classification itself. You might want to consider a different approach than multinomial logit. Latent class models might provide a different, more substantively interpretable fit. There are two broad approaches to LC: Heckman's "time series" and Lazarsfeld's cross-sectional approaches. Both have been built into the Latent Gold software (about $1k)...but there's downloadable, freeware out there too. $\endgroup$ – Mike Hunter Jun 3 '15 at 12:49
  • $\begingroup$ LCA with covariates was my first thought when thinking about the problem. Sadly, the clustered data essentially consists of proportions of word counts (freq divided by all word frequencies per text) and is high-dimensional (100+ features). I resorted to subspace clustering for which I don't know of any program that allows for covariates. That gave me an interpretable classification result with a relatively good fit. I'll try my luck with the multinomial model and if that gives me nothing worthy of consideration, I will try out flattening the data to binary and then using LCA with covariates. $\endgroup$ – quadzar Jun 3 '15 at 13:12
  • $\begingroup$ Ok. How about stepping back from those word counts to their frequency of use in the actual texts, standardizing their frequencies across texts and doing a dimension reduction approach (PCA, CFA or some robust approach) on the transformed data matrix, followed by k-means clustering of the reduced dimensions? That should provide a more insightful classification for analysis than the tf-idf word counts. My opinion is that the tf-idf word counts are aggregated enough that you've lost much of the specificity that would be in the text-level matrix, making it harder for a model to find a signal. $\endgroup$ – Mike Hunter Jun 3 '15 at 13:34
  • $\begingroup$ I thought about standardizing the frequencies, but as the documents have different lengths and I am interested in the relation of words (their frequencies) to one another in a document. Doesn't that get lost when I just standardize and don't normalize somehow (maybe I don't understand what you mean)? Subspace clustering refers to methods where PCA or FA is used during the clustering procedure, which is the only reason why I can estimate these models at all (with R package HDclassif; mclust was only able to estimate the simplest models). By the way, thank you for giving so much feedback. $\endgroup$ – quadzar Jun 3 '15 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.