The domain I'm operating in is Healthcare. So I have a lot of variables like visits to the doctor, prescribed drugs etc. - each one has a time dimension (5-6 years) and a "measure" dimension like count, standard deviation, average. Basically I have a LOT of columns which typically look like the following (e.g. for visits):
"visits_pediatrician_2012_avg", "visits_pediatrician_2013_avg",..., "visits_pediatrician_2016_std" - and then the next doctor specialization comes in.
I hope you get the picture. There are approximately 30k variables. It is important to note that not all of them sit in the same file, otherwise I could just reshape everything (and get a different problem of a lot of observations). I have topic files.
Another important piece of information regarding my problem is that the goal of this project is inference and explanation rather than prediction.
My question is - how can I start analyzing the variables and sorting out which ones are important to me and which ones are not? I read here that some people suggested LASSO or PCA. My problems with those are:
Yes, LASSO is giving me sort of a feature selection eventually, but I totally miss any correlative relationships between 2 or more variables. It might be that
x_1alone gets dropped out by LASSO, but there is important information at the intersection between
x_2that I miss.
PCA will help me with dimension reduction, but then later at the modeling phase, I'll have "cooked" variables which are basically some sort of linear combinations of my original variables. If, for example, my model is logistic regression I wont be able to explain the results because the coefficients I'll get will be assigned to these "cooked" variables which are hard to interpret.
What can be done then to reduce the dimensionality but also to retain explanation ability and not loosing some correlative relationships among two or several more variables in the data?