3
$\begingroup$

Suppose one estimates a linear time series model $$ y_t=\beta_0+\beta_1 x_{t-1}+\varepsilon_t $$ and finds that $\hat\beta_1>0$ and the $p$-value associated with $\hat\beta_1$ is lower than the chosen significance level. Can one say, without any caveats, that

$x$ statistically significantly predicts $y$?

Similarly, can one say

$x$ positively predicts $y$?

My main concern is that the claim about prediction is based on in-sample analysis without stating the implicit assumptions that are required to make a conclusion about out-of-sample results from in-sample results. Another, minor concern is that the claim is based on a significance test of $\beta_1$ rather than a measure of change in prediction errors of $y$ when $x_1$ is added to the model. However, the latter is probably not a problem as the significance of $\beta_1$ probably implies the prediction errors of $y$ will be decreased by use of $x_1$ in the model.

$\endgroup$
1
  • $\begingroup$ “Significantly predicts” means predictions better than random, which is a truly boring hypothesis. Importantly, any threshold for statistical significance is BS. $\endgroup$ Commented Oct 1, 2023 at 12:48

1 Answer 1

0
$\begingroup$

Yes, and Yes. Note that you predict coefficients "only in-sample". To say something about statistical significance, you need a distribution for coefficients. But this is estimated using the in-sample only. You can make a OOS prediction, but technically speaking you cannot say anything about out-of-sample statistical significance.

In your minor concern, I guess you are essentially comparing "$\beta_{1}$" and "$\beta_{1} * x_{1}$". In Statistics it often the relation of concern, not prediction. So as long as $x_{1}$ significantly affects $y$, how much it changes $y$ is of less concern.

$\endgroup$
1
  • 1
    $\begingroup$ What does predict coefficients mean? you need a distribution for coefficients: coefficients are constants in frequentist statistics, and I am not interested in a Bayesian interpretation of this question. You can make a OOS prediction, but technically speaking you cannot say anything about out-of-sample statistical significance. Yes, you can using tests like Diebold-Mariano. how much it changes $y$ is of less concern: my question is specifically about prediction. $\endgroup$ Commented Oct 19, 2019 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.