I'm testing the hypothesis that variable $x$ predicts the variable $y$ AND that it predicts it when adjusted for other variables that have been shown to predict $y$ in the literature ($z_1$ to $z_5$). For this purpose, I built a model that included $x$ and $z_1$, $z_2$, $z_3$, $z_4$, $z_5$ as independent variables.

The dependent variable and $x$ are continuous. The rest of the independent variables are continuous except one that is binary (sex). The dataset includes $27$ cases.

The overall model was not significant ($F=1.90$, $p=0.112$). However, $x$ indeed emerged as a significant predictor ($\beta=0.60$,$t=3.08$,$p=0.007$) while $z_1$ to $z_5$ were not significant. How do you think I should interpret these results?

I know similar questions have been asked about interpreting the results when the $F$ statistic is not significant while a predictor is significant. However, in this case, the significant predictor is not just one of the variables included in the model, but it is the one that is being specifically tested with an a priori hypothesis.

  • $\begingroup$ You might find the answer here or here. $\endgroup$
    – Durden
    Commented Feb 4 at 17:09
  • 2
    $\begingroup$ Please edit the question to provide more detail about the data: how many observations and what the natures of the predictors and outcome are (e.g., continuous, binary, ordinal, multi-level categorical). For example, you might not have had enough observations to fit a 6-predictor model reliably, particularly if some of those were multi-level categorical predictors or continuous predictors fit with polynomials or splines. $\endgroup$
    – EdM
    Commented Feb 4 at 18:01
  • $\begingroup$ @EdM. Thank you. Just edited to include the number of cases and data types. $\endgroup$
    – CagYuk
    Commented Feb 4 at 18:06
  • 3
    $\begingroup$ With 27 cases, your full model is much too full. You have a big risk of overfitting. $\endgroup$
    – Peter Flom
    Commented Feb 4 at 18:35
  • 1
    $\begingroup$ Sure, it's not many observations. A fairly standard approach in economics and finance regression applications would be to provide three regression results Y on X; Y on Z1-Z5; and Y on X and Z1-Z5, in a table. Emphasis being on the coefficient estimates and their p-values. Gives the reader more information. $\endgroup$ Commented Feb 4 at 21:02

1 Answer 1


Your question is very similar to this question, except that you had a prior hypothesis about the association between $x$ and $y$. As this answer to that question points out, having too many predictors in a model can end up in a situation where the overall F-test becomes "statistically insignificant" even while the coefficient for the predictor specifically associated with the outcome maintains its individual "significance."

As this answer to one of the linked related questions points out, in this situation you could do an F-test that compares a model including $z_1$ though $z_5$ as predictors, but without $x$, against the model that includes all the predictors. That's a direct test of the pre-specified hypothesis that $x$ improves the model based on the $z$'s alone. The result should be essentially what you found for the coefficient for $x$, but framing the comparison that way helps emphasize your pre-specified hypothesis.

I'd be very wary, however, of putting much faith in results from models with only $27$ observations and $5$ or $6$ predictors. There are many ways that things can go wrong. It's hard to check how well the OLS assumptions are met. Results of standard OLS models with fewer than about $15$ observations per estimated coefficient can depend heavily on the vagaries of your particular data sample and might not extend well to new data samples. A single outlier could easily lead to problems. Look at some of the highly rated entries on this site having to to with overfitting.

Furthermore, it's seldom true that a continuous predictor is exactly linearly associated with a continuous outcome. It's good to allow for the model to fit continuous predictors flexibly. Frank Harrell's Regression Modeling Strategies, a valuable and freely available resource, goes into that in Chapter 2. Such flexible fitting, however, requires estimating more than 1 coefficient per continuous predictor and thus requires even more observations, unless you use penalized methods like ridge regression or some types of generalized additive models.

  • $\begingroup$ Thank you so much Shawn for this detailed answer. $\endgroup$
    – CagYuk
    Commented Feb 5 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.