I am running a multiple linear regression model and I have 8 covariates, 4 of them are highly correlated (r>0.7). So I created z scores and then created a composite. When I re-ran the model with this composite, my predictor p-value became significantly larger and my R2 went down. Why is this happening? I thought p-values decreased after accounting for multicollinearity?
When you create a composite, you lose information. Therefore, it seems likely that p values will go up and $R^2$ will go down, although I don't think this is necessarily always the case.
You shouldn't rely on correlations to test collinearity; use condition indexes or VIFs (I prefer the former) and, if you do find collinearity, creating a composite is only one option. You could try ridge regression or elastic net instead.