I am performing a multiple linear regression with response, Y and independent variables X1, X2 and intercept term.
The regression coefficients indicate statistically significant prediction of both X1 and X2 on Y (p<0.05). however, when I perform some model validation, I get the following:
- Linear relationship b/n predictors and response - no
- Normality of residuals - approximately (see figure)
- Collinearity of predictors - yes
- Auto-correlation of residuals - no (see figure)
- Heteroskedasticity of residuals - yes (see figure)
given these violations (1,3 and 5), I am obviously concerned about inflated Type 1 error rate.
Hence, I estimated empirical null distribution of regression co-efficients by repeating regression multiple times (500), each time with a randomly resampled (without replacement) version of the response vector. after this, the pattern of results remain similar, with lower significance levels (i.e. higher p-values).
My questions are:
- is this non-parametric hypothesis test a valid way of accounting for violations of the regression model?
2a. a slightly 'philosophy of data analysis' question: what can I infer from the significant effects on the non-parametric test? I assume I can infer that the predictors X1 and X2 do in fact influence the response Y.
2b. but what if I suspect that a hierarchical regression framework (i.e. multi-level framework) is better suited to modelling the response Y? does the significance of the reg. coeffs. imply anything about suitability of the regression modelling framework? if not, and if the regression model framework is in fact not the correct one, do the reg. coeffs. mean anything at all (even though they are significantly non-zero)?
any thoughts welcome!