Linear regression withouth intercept R

Question

I'm trying to implement a cost function for linear regression withouth intercept.

I tried understanding what "no intercept" means, but not found anything which could reassure me. I have a dataset that I scale in R with scaled_data <- scale(data).

As I'm scaling, does this mean that my linear regression does not rely on an intercept?

Answer 1

The formula of the linear regression is,

$$y=\beta_0+\beta_1x_1+\ldots+\beta_px_p$$

where the coefficient $\beta_0$ is the intercept in the model. This can be written in matrix notation as,

$$y = X\beta+\varepsilon$$

where we are making a slight abuse of notation, because in order to include the $\beta_0$ in this formula, we are writing the vector $$\beta=(\beta_0,\beta_1,\ldots,\beta_p)$$ and we are writing the matrix $$X=(1,x_1,\ldots,x_p)$$

So if you want to implement a cost function for a linear regression model without intercept, you just need to remove $\beta_0$ from the vector $\beta$ and remove the vector $1$ from the matrix $X$.

Now, using the already available implementation of linear regression in R, this can be done, as suggested in the comments, by setting

lm(cost ~ -1 + predictor1 + predictor2, data)

The -1 there means that you do not want an intercept in the model. The result of this however is very different from the one you would obtain if you scale the data.

Scaling the data means that your data is centered, and the intercept of models built using centered data is $0$. In this case, you would obtain the same result with a non intercept model.

But if you build a model without an intercept in a dataset that is not scaled, you will force your regression line to go through the value (0,0), and then the result of a model with intercept and one without intercept will be different.