1
$\begingroup$

I notice when running regressions in programs such as Stata that the constant term has a p value. Why and what does this p value represent?

$\endgroup$
  • 2
    $\begingroup$ Why should it not? $\endgroup$ – Tim Jan 5 at 19:40
2
$\begingroup$

The standard error, p-value, and confidence intervals about regression constants represent the uncertainty in the estimated value of the constant.

Let's take bivariate OLS linear regression as an example. One of the primary questions we answer with linear regression is what straight line* best predicts the trend in mean values of $y$ over values of $x$?

When we recall that a straight line is defined by two parameters, a $y$ intercept, and a slope describing change in $y$ given a 1 unit increase in $x$, we can recognize the regression constant term (let's call it $\beta_{0}$) as the former, and the coefficient of your predictor variable (let's call it $\beta_{x}$) as the latter, we can then see one place where statistical inference enters a linear regression: what are the estimated values of $\beta_{0}$ and $\beta_{x}$, and what are the uncertainties about these estimates?

The question mentioned p-values, and that specifically means hypothesis tests are involved, so setting aside confidence intervals, what hypotheses are being tested? These are:

  1. $H_{0}: \beta_{0} = 0\text{, with } H_{\text{A}}: \beta_{0} \ne 0$; and
  2. $H_{0}: \beta_{x} = 0\text{, with } H_{\text{A}}: \beta_{x} \ne 0$

You can see that hypothesis (1) gives some interpretability to a rejection decision attached to the p-value for a regression constant: is there evidence that the intercept has a different value than 0? If a p-value is low enough (with respect to $\alpha$), then the answer is yes, otherwise the answer is there is not evidence that the intercept has a value different than 0. Aside: this is, of course, different than asking if there is evidence that the intercept is equivalent to 0 (say, by using TOST in Stata with the tostregress command which is part of the tost package).

As an example, let's look at some Stata regression output, where we regress automobile model weight onto automobile length:

. sysuse auto
(1978 Automobile Data)

. regress weight length

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(1, 72)        =    613.27
       Model |  39461306.8         1  39461306.8   Prob > F        =    0.0000
    Residual |  4632871.55        72  64345.4382   R-squared       =    0.8949
-------------+----------------------------------   Adj R-squared   =    0.8935
       Total |  44094178.4        73  604029.841   Root MSE        =    253.66

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      length |   33.01988   1.333364    24.76   0.000     30.36187    35.67789
       _cons |  -3186.047   252.3113   -12.63   0.000     -3689.02   -2683.073
------------------------------------------------------------------------------`

We can see that the estimated intercept term ($\hat{\beta}_{0}$) is –3186.047, and that the reported p-value is less than 0.001. Therefore, with an $\alpha = 0.05$, we would reject $H_{0}: \beta_{0} = 0$ in favor of $H_{\text{A}}: \beta_{0} \ne 0$, and conclude that we found evidence that the intercept was significantly different (less) than zero, at the $\alpha=0.05$ level. Aside: remember that what we want to make an inference about are $\beta_{0}$ and $\beta_{x}$, but what we actually have to work with are $\hat{\beta}_{0}$ and $\hat{\beta}_{x}$, so we need statistical inference to get to the former from the latter.

One final point: I began this answer by assuming "OLS least squares." It may be obvious that this answer also applies to multiple OLS linear regression where there is more than one predictor variable. But it is worth saying that this also applies to other forms of regression, including regressions with nonlinear link functions like logistic regression, to generalized linear models, to nonlinear least squares regression, etc: whenever you are estimating parameters (whether or not those parameters are model constants), you have the opportunity to perform statistical inference about the values of those parameters. Aside: while the model constant term in these examples defines the $y$ intercept of a line, and, for example, defines the $y$ intercept of the sigmoid logistic curve in logistic regression, in some models (notably autoregressive time series models) there may not be any $y$ intercept per se, even if there is a model constant term estimated—in other words the meaning of $\beta_{0}$ may not always be an intercept term. Second aside: while your question is about frequentist statistical inference, Bayesian statistical inference also cares about constant terms.


* Or at least a straight line segment over observed range of $x$ values.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ When the p value of the variable is non significant, but the p value of the constant is highly significant...What does this mean? There is no significant relationship between the two variables, but if there was, we would be able to predict it with significance? $\endgroup$ – Paze Jan 5 at 19:23
  • $\begingroup$ @Paze Hi! I am saving the question in parts as I write it. I think the question in your comment is answered now... I have a few more additions to make, so even more questions may get answered. :) $\endgroup$ – Alexis Jan 5 at 19:33
  • $\begingroup$ @Paze Alright, I am all done answering the question know. :) By the way, welcome to CV! $\endgroup$ – Alexis Jan 5 at 20:06
  • $\begingroup$ @Paze And in just in case here's an even more explicit answer to the question in your comment: a non-significant slope ($\hat{\beta}_{x}$) means there is not evidence that the slope is different than 0. Another way of saying that is there is not evidence that the regression line is different than a horizontal line (i.e. $y$ does not change over values of $x$). However, even a horizontal line has a $y$ intercept. $\endgroup$ – Alexis Jan 5 at 20:18
4
$\begingroup$

The intercept $\beta_0$ is interpreted as the predicted value of $y$ when $x$ is zero. The p-value corresponds to the test of $H_0: \beta_0 = 0$. Sometimes this is an interesting test, and other times it isn't. In general, you might ask the question, "For a given value of $x$, does the predicted value for $y$ differ from zero?". The test of $\beta_0$ answers this question for $x=0$. If you subtract a value $k$ from all of your $x$ values and then rerun the regression, the test of the intercept now answers this question for $x=k$.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.