7
$\begingroup$

When using statistics software, When defining your linear models, why is the intercept typed in as a 1, rather than "const" or "intercept" or something. What significance does 1 have?

Is there some historic reason? Or is this logical in some way I am failing to grasp? The intercept could very well be any number.

Example from statsmodels library in python:

model = smf.ols('Height ~ 1', data = height_sample_data)

I know lmer package for R is very similar.

$\endgroup$
  • 5
    $\begingroup$ The intercept is the coefficient (which indeed could have any value), but what you enter into the regression program when you fit the model are not the coefficients, but the things you multiply the coefficients by in the regression equation (the $x$'s). What do you multiply the intercept by in the regression equation? (Note that $\beta_0 \times 1 = \beta_0$.) $\endgroup$ – Glen_b Mar 19 at 4:31
16
$\begingroup$

It is logical, once you consider the matrix notation that your formula will be translated into internally. In the matrix, the non-constant predictors will be translated into (one or more) columns, and the intercept will be translated into a column consisting entirely of ones.

For instance, in R you would write a very simple OLS as:

lm(z~1+x+y)

In matrix notation, this would be translated into a model

$$ \begin{pmatrix} z_1 \\ z_2 \\ \vdots \\ z_n \end{pmatrix} = \begin{pmatrix} 1 & x_1 & y_1 \\ 1 & x_2 & y_2 \\ \vdots & \vdots & \vdots \\ 1 & x_n & y_n \end{pmatrix} \begin{pmatrix} \beta_0 \\ \beta_x \\ \beta_z \end{pmatrix} +\begin{pmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{pmatrix}, $$

and now you see where the $1$ comes from.


Actually, you could leave the 1+ out, since R will always presume you want to include an intercept, so this is completely equivalent to

lm(z~x+y).

However, if you want to suppress the intercept, you would write something like

lm(z~x+y-1),

which would be translated into a matrix without a 1 column:

$$ \begin{pmatrix} z_1 \\ z_2 \\ \vdots \\ z_n \end{pmatrix} = \begin{pmatrix} x_1 & y_1 \\ x_2 & y_2 \\ \vdots & \vdots \\ x_n & y_n \end{pmatrix} \begin{pmatrix} \beta_x \\ \beta_z \end{pmatrix} +\begin{pmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{pmatrix}, $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.