So the context I am working with is defining signatures to classify different cell types (I have 7) in order to then use with statistical deconvolution by regression. So far, I have been using a multinomial GLMnet with elastic net penalties. While the features thus selected generally do a great job at informing regression based estimation of frations, there are some cell types for which this works very poorly.
I want to tune the model / pick features that are better able to differentiate between CD4 (vastly underestimated) and CD8 (vastly overestimated) while retaining the performance on the other cell types. What is the best way to optimise my tuning workflow to achieve this (I work within R)?
My options are
a) Use pairwise comparisons, then use nonzero coefficients for each pair to inform the signature matrix. However in each class I have around 4 samples and no more.
b) Use a method that implictly takes into account the hierarchical nature of the data, with some cell types being more correlated than the others, and falling into different clusters. Are there any machine learning methods specifically suited for this?