What is the best way/method to model patterns across coefficients and reduce number of coefficients based on these patterns?
We have hundreds of regressors on the same scale and try to reduce the number of coefficients with respect to the regressors' relationship with our outcome variable.
To some extent we try to invert the logic behind finite mixture regression: Instead of estimating different coefficients for different subgroups of our observations, we try to built subgroups of regressors which have a similar effect and are interested in the "overall coefficients" of these subgroups of regressors. Our idea is to estimate both membership to the latent regressors and the final regression parameters for the latent regressors simultaneously and to allow for inference in the end. Our thought was as follows: While a finite mixture regression assumes a mixture distribution of the outcomes and estimates the membership of the observations to the distributions and the regression parameters simultaneously, we rather wanted to transfer this idea to the parameter vector. Instead of assuming a multivariate normal distribution of the regression parameters, our idea was to assume a multivariate mixture distribution at this point, so assign regressors to certain latent regressors and somehow estimate the overall effect for each latent regressor on our outcome.
One easy example would be the following:
Assume we have 500 regressors in total. 250 regressors have a positive coefficient around 100. 250 regressors have a negative coefficient around -100.
Assuming n "latent regressors", our model should (1) identify the regressors which have similar effects on the outcome (in the example of 2 latent regressors, the first 250 regressors should be identified as one latent regressor due to their positive effect, the next 250 regressors should be identified due to their negative effect), (2) should consider the original regressors as n latent regressors in the model and (3) should also give n coefficients and allow for inference for the latent regressors.
Is there any appropriate method? Since we do not come from econometrics in the first place, we unfortunately do not know whether our idea is realistic at all. PLSR that Peter mentioned may indeed be the best way and we have to rethink the method.
