I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of features as independent variable?.
For example Instead of: Y ~ X1 + X2 + X3 + Z1 + Z2 + Z3
Can I use Y ~ X1 + X2 + X3 and Y ~ Z1 + Z2 + Z3
then use the fitted value Y1 and Y2 from those two models to do: Y ~ Y1 + Y2
Please correct me if there is any issue with this method. Thanks!
I think what you are saying is that you have two types of columns... type 1 and type 2. You have a target variable $y$ to estimate. What you are suggesting to do is to estimate $y_1$ with the type 1 columns, then $y_2$ with the type 2 columns, and finally estimate $y$ as a function of $\hat y_1$ and $\hat y_2$.
Although this is certainly possible, you don't indicate why would you be interested in doing this. If you want to do dimensionality reduction, I would recommend you to do PCA instead, and then regress $y$ into the principal components.
The problem of your approach is that you are introducing unnecessary noise for the estimation of the parameters.
