I wish to create a composite variable of a number of highly correlated variables. Each one of these variables contains different information and is useful in it's own right. But they are all highly correlated (sometimes 0.8+). Extreme values of overfitting regularly occur during linear regression with one parameter becoming highly negative and another one becoming highly positive, such that the correlations overall net out the two parameter effects.
What is this recommended approach to doing regression on highly correlated variables while avoiding issues such as this caused by correlation?
Previously I was constraining all coefficients to be positive or zero. However I am moving things over to R and only a limited number of packages seem to support constraints. So I wanted to know if there are better ways of approaching this? Regularisation or selection would probably solve some of the problem, although they wouldn't rule out some spurious values coming through because of the correlations. Another possibility I thought of was using PCA with all components being used in the regression?
If it's of interest, one of the examples would be using financial data to create a composite Value indicator using PE ratios, P/CF ratios and P/Sales ratios, each of which contains different but closely related information. It would add another couple of ratios in, and include relative to industry and absolute versions of each, so I am looking at about 12 or so different variables with correlations often between 0.8 to over 0.9.