How to interpret an insignificant intercept in a model with significant coefficients?

Question

Imagine a regression with one dependent variable (y) and two explanatory variables (x1 and x2).

Both coefficients of x1 and x2 are significant. But the intercept is around 0.1 is not significant. How to interpret this situation? The intercept must be y when both x1 and x2 = 0. If it is insignificant, it must mean that it can be concluded, at a certain significance level, that it is different from 0?

Imagine I add another variable to my model (x3). Now, the intercept changes to lets say 5 and becomes significant. What is going on here? How can x3 alter the intercept so much?

Answer 1

The estimated parameters are random variables and follow a certain distribution. Imagine that you make the hypothesis that the parameter you are interested in (intercept) is equal to zero. You can build a distribution around this value (with a given mean and variance) usually given by the theory (for example OLS estimator are normally distributed under the assumption of normal error terms, or the Maximum Likelihood estimators are asymptotically normal etc). If the estimated parameter, say the intercept, lies within the $(1-\alpha)\%$ of the distribution under the null (where $\alpha$ is the size of the test, usually 0.05), then you cannot conclude that the estimated parameter is statistically different from zero.

In other words, your estimated parameter is statistically different from zero only if the estimated parameter lies in a region of the distribution whose associated probability is $<\alpha$, under the null hypothesis that the true population parameter is equal to zero.

If the intercept is not significantly different from zero, then it means that if $x_1$ and $x_2$ are zero, also $y$ is equal to the intercept.