# Tag Info

## Hot answers tagged intercept

39

The Elements of Statistical Learning by Hastie et al. define ridge regression as follows (Section 3.4.1, equation 3.41): $$\hat \beta{}^\mathrm{ridge} = \underset{\beta}{\mathrm{argmin}}\left\{\sum_{i=1}^N(y_i - \beta_0 - \sum_{j=1}^p x_{ij}\beta_j)^2 + \lambda \sum_{j=1}^p \beta_j^2\right\},$$ i.e. explicitly exclude the intercept term $\beta_0$ from the ...

30

$\beta_0$ is not the odds of the event when $x_1 = x_2 = 0$, it is the log of the odds. In addition, it is the log odds only when $x_1 = x_2 = 0$, not when they are at their lowest non-zero values.

29

It will almost never be meaningful to use the no intercept model in logistic regression. The intercept parameter $\beta_0$ is modelling the marginal distribution of the response $Y$, so using $\beta_0=0$ is tantamont to assuming that $P(Y=1)=0.5$, marginally. Do you really know that? If that is untrue, you cannot trust any inference from the no intercept ...

21

It's unusual to not fit an intercept and generally inadvisable - one should only do so if you know it's 0, but I think that (and the fact that you can't compare the $R^2$ for fits with and without intercept) is well and truly covered already (if possibly a little overstated in the case of the 0 intercept); I want to focus on your main issue which is that you ...

19

This is a consequence of Jensen's Inequality. You want $E[y|x]$, but exponentiating the predicted value(s) from the log model will not provide unbiased estimates of $E[y|x]$, as $$E[y_i|x_i] = \exp(x'\beta) \cdot E[\exp(u_i)]$$ and the second term is omitted in your calculation. If the error term $u \sim N[0,\sigma^2]$, then $E[\exp(u)] = \exp(\frac{1}{2}\... 18 Adding +0 (or -1) to a model formula (e.g., in lm()) in R suppresses the intercept. This is generally considered a bad thing to do; see: When is it OK to remove the intercept in lm()? When forcing intercept of 0 in linear regression is acceptable/advisable The estimated slope is calculated differently depending on whether the intercept is estimated ... 16 It is logical, once you consider the matrix notation that your formula will be translated into internally. In the matrix, the non-constant predictors will be translated into (one or more) columns, and the intercept will be translated into a column consisting entirely of ones. For instance, in R you would write a very simple OLS as: lm(z~1+x+y) In matrix ... 14 Here is an illustration that simulates$y$and$x$independently of each other so that the true slope is zero. The mean of$y$is nonzero, such that the true intercept is also nonzero. The LS line without intercept must start at$(0,0)$without intercept, and will try to "catch up" with the data points as quickly as possible if$y$has nonzero mean, which ... 13 The coefficients of each predictor are almost always going to change when you add more predictors. This is an example of the answer changing when you ask a different question. Your software should let you fit a regression with no predictor at all. For example, if I try to predict people's weights with a regression with no predictors, then I will get the mean ... 12 Short answer to question in title: (almost) NEVER. In the linear regression model $$y = \alpha + \beta x + \epsilon$$, if you set$\alpha=0$, then you say that you KNOW that the expected value of$y$given$x=0$is zero. You almost never know that.$R^2$becomes higher without intercept, not because the model is better, but because the definition of$...

11

If you write out the fitted model for the log odds of smoking $$\log \frac{\Pr(Y=1)}{\Pr(Y=0)} = -4.380\,1 + -0.324\,56\ I_\mathrm{teen} + 1.451\,19 \ I_\mathrm{mature} + -0.989\,1\ I_\mathrm{old}$$ where the dummies are $$I_\mathrm{teen}=\left\{ \begin{array}{l l} 0 & X\neq\mathrm{teenager}\\ 1& X=\mathrm{teenager}\\ \end{array}\right.$$ &c., ...

11

This an example of linear regression fit. The intercept of this fit is negative and it fits well.

11

In addition to @DaveT's helpful answer, here are a few more clarifications regarding the estimated intercepts in your models. Model 1 The (true) intercept in your first model lm(mpg ~ 1, data=mtcars) represents the mean value of mpg for all cars represented by the ones included in this data set, regardless of their displacement (disp) or horse power (hp)...

11

I will give you a run-down of the terminology used in statistics, which I think is sensible terminology. Cases (1) and (2) do refer to bias in the usual statistical sense, (4) refers to something closely related, and (3) is just misleading renaming of an object that already has a perfectly sensible name. Just to quickly dispose of (3), I note that the term ...

10

You need to drop the intercept (the vector of 1's) from the mm matrix since the glmnet package automatically demeans the data and reports the intercept term by default. Alternatively, you can use the intercept parameter to glmnet (TRUE by default).

10

Start with a simple logistic regression: $\,\,\text{logit}(\mu) \,= \beta_0 + \beta_1 x\quad\quad$ (original) $\quad\quad\quad\quad= \beta_0^* + \beta_1^* (x-\bar{x})/s_x\quad\quad$ (standardized x) $\quad\quad\quad\quad= (\beta_0^* -\beta_1^*\bar{x}/s_x)+ (\beta_1^*/s_x) x$ So $\beta_1=\beta_1^*/s_x$ and $\beta_0=\beta_0^* -\beta_1^*\bar{x}/s_x$ More ...

10

The formula lm(formula = y ~ x1 + x2) will include an intercept by default. The formula lm(formula = y ~ x1 + x2 -1) or lm(formula = y ~ x1 + x2 +0) is how R estimates an OLS model without an intercept. The formula lm(formula = y-1 ~ x1 + x2) estimates a model against a dependent variable y with 1 subtracted from it. Centering all terms at their ...

10

When dealing with categorical variables in LASSO regression, it is usual to use a grouped LASSO that keeps the dummy variables corresponding to a particular categorical variable together (i.e., you cannot exclude only some of the dummy variables from the model). A useful method is the Modified Group LASSO (MGL) described in Choi, Park and Seo (2012). In ...

9

The intercept in a linear regression model may represent two totally different things: A) Your theoretical model may lead you to a specification with a constant term. A basic example from Economics is when one wants to estimate the parameters of a production function (a statistical relationship that links output produced with production factors used) ...

9

The intercept should generally only be omitted if all the predictors and the response have mean=0 (in which case the intercept must necessarily be 0). Setting standardize=TRUE, which is the default option for glmnet::glmnet, only standardizes the predictors. The function has another parameter to standardize the response, but by default this is set to ...

8

Most multiple regression models include a constant term (i.e., the intercept), since this ensures that the model will be unbiased--i.e., the mean of the residuals will be exactly zero. (The coefficients in a regression model are estimated by least squares--i.e., minimizing the mean squared error. Now, the mean squared error is equal to the variance of the ...

8

The intercept has a meaning here, as in any regression. But the meaning is neither interesting nor useful. As calendar year is the predictor, the intercept here is the value predicted for year 0. Set aside the fact that there was no year 0 and years are reckoned in retrospect, in the calendar you are using, to have run ..., 1 BC (or BCE), 1 (AD), etc. ...

8

Something like this should do it: fit <- lm( I(y-9.81) ~ 0 + x1 + x2 + I(x3^2) + x4 + x5 + x6 , data=data[i:(i+k),]) Something similar should be possible in many packages. An alternative: interc <- rep(9.81,k+1) fit <- lm(y ~ 0 + x1 + x2 + I(x3^2) + x4 + x5 + x6 + offset(interc),data=data[i:(i+k),]) While the coefficients and standard errors ...

8

Your characterization of how multiple regression works is inaccurate. Your version implies fitting a simple linear regression for each variable in turn (and presumably using each of those slopes as the coefficient for that variable in the multiple regression model). This notion leaves you with the problem of how to deal with the fact that the intercepts ...

8

The formulas are the same as always, so let's focus on understanding what's going on. Here is a small cloud of points. Its slope is uncertain. (Indeed, the coordinates of these points were drawn independently from a standard Normal distribution and then moved a little to the side, as shown in subsequent plots.) Here is the OLS fit. The intercept is ...

8

Nick Cox provided an excellent response and I wanted to add a more intuitive answer. Model 1 Model 1 investigates the relationship between IQ and Brain size among subjects represented by the ones in the study, regardless of those subjects' Gender, Height and Weight. In other words, if you imagine the target population of subjects from which the subjects in ...

7

As mentioned before it is sort of hard to justify not using an intercept unless there is strong knowledge that the linear regression line passes through the origin. However, how fitting the model with and without the intercept affects the residuals is kind of case by case. For example, if the true model that generated the data did have an intercept far ...

7

Theoretically, does the intercept term in a logistic regression model capture all unobserved effects? This is an interesting question, and I can see how with some simple experiments, one might think that this is the case. In fact, in my first attempt to set this up, I actually created a demonstration would only incorrectly estimate the intercept when I mis-...

7

@kjetil b halvorsen has done a good job outlining the main ideas here. Let me add a couple supplementary points. With a categorical variable, suppressing the intercept results in level means coding, instead of the default reference level coding. I explain this in greater detail here: How can logistic regression have a factorial predictor and no ...

7

This question touches on a number of existing posts but I didn't find one that related to all of it. You should not include the constraint as if it were an ordinary data point, with the same uncertainty (but see point 4.) ordinary regression through the origin is entirely straightforward. Consider $y_i = \beta x_i + \epsilon_i$ \$S = \sum_i (y_i - \beta ...

Only top voted, non community-wiki answers of a minimum length are eligible