We're trying to build a statistical model that predicts hand weight based upon measurements derived from photos of hands. There are about 50 individual measurements per hand, but the measurements are obviously highly correlated. For example, tip of ring finger to wrist is obviously highly correlated with tip of middle finger to wrist, etc. Collinearity diagnostics are obviously off the charts.
We tried to simplify the model by doing a PCA to cluster the individual measurements so we didn't have 50 highly correlated predictors. We ended up with 5 PC's but most of the variables loaded onto only one PC (assuming due to the very high correlation between the measurements).
The real concern here is overfitting of our model. The difference between the R-Squared and Adjusted R-Squared is quite significant no matter how we run the model.
Does anyone have any advice for me? Maybe there's some ways of dealing with the data that I'm not aware of (something other than straight forward multiple regression like PLS)? We have a pretty decent sample size (n = 800) and we're mainly concerned with predicting hand weight using the 50 measurements of the hands. Both the predictor and outcome variables are continuous.