1
$\begingroup$

What can I do when the p-value of my regression is not significant? I tried to transform it with log or sqrt, it improved a little, but not enough to go below the 5%.

The residuals follow a Normal distribution and there is no strong correlation between the variables.

enter image description here

Does it mean the linear regression just doesn't fit my data or, because the p-value is close to the threshold, I can ignore it?

Edit : I did a cumulative coding because there were lots of categorical and ordinal variable, that explain the difference between the R^2 and ajusted R^2, see below the functionnement (not my data, just for the explanation)

insignificant

$\endgroup$
22
  • 10
    $\begingroup$ You are approaching this with the wrong mindset. Your goal is not creating a significant p-value. $\endgroup$
    – Roland
    Apr 25, 2022 at 10:36
  • 4
    $\begingroup$ I agree with @Roland. Further, it looks like you're estimating a model with 15 parameters and 24 observations - that's asking the data to do an awful lot. You see that reflected in the huge difference between the multiple R-squared and the Adjusted R-squared. $\endgroup$ Apr 25, 2022 at 10:44
  • 1
    $\begingroup$ A few things. 1. The insignificant F-statistic means that all model parameters are not jointly different from zero - that all non-zero estimates could have arisen by chance rather than a systematic relationship that you've uncovered. I wouldn't pay as much attention to the R-squared value as the difference between the R-squared and adjusted R-squared values, which is huge - indicating several "irrelevant" variables in the model. $\endgroup$ Apr 25, 2022 at 12:10
  • 2
    $\begingroup$ With only 24 observations you should only be trying to fit about 2 predictors; otherwise you run risks of overfitting or even of missing true associations with outcome because you don't have enough data to support a "significant" F-test. See Frank Harrells course notes on Regression Modeling Strategies, especially Chapter 4. You should step back and think about the specific question you want to answer and whether you have enough data to answer it reliably. $\endgroup$
    – EdM
    Apr 25, 2022 at 13:40
  • 3
    $\begingroup$ So why are you obliged to pursue a bad regression strategy? $\endgroup$
    – Dave
    Apr 25, 2022 at 14:47

1 Answer 1

3
$\begingroup$

One interpretation of a high p-value in a regression model is that your model doesn't produce better predictions than a model that left out everything except the intercept.

This can happen if some of the predictors you used are unrelated or only weakly related to the response. Estimating coefficients for those predictors makes your other estimates worse. In your case, you are including 15 predictors in a dataset of 24 observations, so it shouldn't be surprising that you can't get good estimates of all 15 coefficients.

The usual solution to this is to simplify your model. Running all possible subsets of your predictors can find the model with the best p-value, but will also make your p-value artificially small. (The selected F statistic won't have an F distribution under the null hypothesis.) There are several strategies for dealing with this, all with pitfalls, so automatic selection is likely to be unreliable; you should consult a good regression text or a local expert.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.