I'd like to use regularized logistic regression to classify elements (DNA sequences, i.e., character strings) into one of two categories based on the presence/absence (1/0) of many (k=50) features (DNA sequence motifs, i.e., substrings). I have approximately 20000 elements, which are split approximately 90:10 between my two classes.

I expect there to be significant interactions among some of these features (I mean non-additive effects between specific pairs of features), so I believe I want to include interaction terms in my model. Doing this exhaustively would mean examining a large number of interactions (50 choose 2=1225). To me, this seems bad--I don't think I have enough data to accurately fit a model with so many terms, and I would expect fitting such a model would be computationally challenging.

If I were working with a smaller number of features, would it be best to fit a model with all possible interactions, and then perform a post-hoc analysis of which terms significantly improve the model? For my current case, is it valid to pre-filter for specific interactions of interest (by identifying pairs of features that are enriched in (cooccur more often in) one class of elements vs. the other)? Thanks for your input.

  • $\begingroup$ What about kernel logistic regression with perhaps a polynomial kernel? $\endgroup$
    – jld
    Commented Aug 5, 2015 at 14:02
  • $\begingroup$ Roughly how many such elements/ observations do you have? $\endgroup$
    – dardisco
    Commented Aug 6, 2015 at 1:35
  • $\begingroup$ Edited post to clarify # of elements/split between classes. $\endgroup$
    – pseudo
    Commented Aug 6, 2015 at 14:03
  • $\begingroup$ What about using best subset-selection or shrinkage (i.e. ridge regression or lasso): stats.stackexchange.com/questions/127444/… $\endgroup$
    – Dan
    Commented Aug 6, 2015 at 14:08


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.