Dear Cross Validated Community,
Our lab is trying to develop a classification-type model that categorizes chemicals into one of three groups using data from cell culture experiments. We have been using Multinomial Logistic Regression but have some questions about the appropriateness of our approach that does not seem to be addressed by online help, or in text books or online material that we are aware of.
Let's say:
The three Categories we're trying to predict = A, B or C
Measured Responses = W, X, Y and Z
Training Set Chemicals = 1 - 68
Concentrations = a - t
Cells were treated with each of 68 chemicals. These first 68 are training set chemicals, each with an associated a priori classification as to Category A, B or C.
Every chemical was studied at multiple (a - t) concentrations, and each concentration was studied in a single well (that is no replicate treatments).
Univariate analyses show Measured Responses W, X and Y are predictive of Category A, B or C.
The 3 factor model W, X and Y is better than any one or two factors models in terms of predicting category A, B or C.
For the logistic regression model above, we originally used one "equitoxic concentration" across the 68 chemicals. So each of 68 chemicals provided one W, X and Y value. Some scientists in our field suggested it would be more ideal to consider "all concentrations" in the model as opposed to reducing the data set down to one equitoxic concentration. With this in mind, we have more recently used all concentrations, a - t, in our three factor model, meaning each chemical is providing 20 W, X and Y responses.
In this scheme, where 20 W, X and Y responses are included for each training set chemical, we a priori specify Category A, B or C for each of the 20 concentrations studied. That being said, W, X and Y responses tend to show better discrimination between categories with increasing cytotoxicity (Measured Response Z), therefore we have begun using a three factor model W, X and Y that is weighted on Z. (JMP 12's "weight" feature in Logistic Regression platform.) This results in better R^2, ROC and Wald statistic values compared to an unweighted three factor model.
Cross validation using leave-one-out suggests the three factor model weighted on Z is working well in regard to predicting class A, B or C. A typical example when using this scheme: a chemical will tend to exhibit low probability scores for any one Category over for instance the lowest 10 concentrations tested (while Z, cytotoxicity, is low). However as Z increases the probability scores increase where they often exceed 90% for a single Category. We take this value (exceeding 90% probability) as the model's prediction as to Category A, B or C. So in this manner we seem to be able to characterize the entire dose-response relationship, as a chemical transitions from having little to no biological effects to exerting clear responses that are indicative of Category A, B or C.
While the model is working well, we are unsure about all the assumptions implicit in the Multinomial Logistic Regression platform. Are we violating assumptions by having each chemical provide 20 W, X, Y and Z values? Or is this acceptable?
Interested in your opinions about our approach, and alternate advice as necessary.
