Questions tagged [intercept]

The intercept in regression-type models is the value of the Y variable when all X variables are 0.

Filter by
Sorted by
Tagged with
139
votes
9answers
139k views

When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the ...
116
votes
2answers
58k views

Removal of statistically significant intercept term increases $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
32
votes
3answers
49k views

Why are bias nodes used in neural networks?

Why are bias nodes used in neural networks? How many you should use? In which layers you should use them: all hidden layers and the output layer?
24
votes
8answers
57k views

When forcing intercept of 0 in linear regression is acceptable/advisable [duplicate]

I have a regression model to estimate the completion time of a process, based on various factors. I have 200 trials of these processes, where the 9 factors being measured vary widely. When I perform a ...
22
votes
3answers
15k views

Importance of the bias node in neural networks

I'm curious to know how important the bias node is for the effectiveness of modern neural networks. I can easily understand that it can be important in a shallow network with only a few input ...
22
votes
1answer
7k views

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them ...
21
votes
5answers
11k views

Reason for not shrinking the bias (intercept) term in regression

For a linear model $y=\beta_0+x\beta+\varepsilon$, the shrinkage term is always $P(\beta) $. What is the reason that we do not shrink the bias (intercept) term $\beta_0$? Should we shrink the bias ...
20
votes
3answers
9k views

Why would one suppress the intercept in linear regression?

In a number of statistical packages including SAS, SPSS and maybe more, there is an option to "suppress the intercept". Why would you want to do that?
16
votes
3answers
54k views

Intercept term in logistic regression

Suppose we have the following logistic regression model: $$\text{logit}(p) = \beta_0+\beta_{1}x_{1} + \beta_{2}x_{2}$$ Is $\beta_0$ the odds of the event when $x_1 = 0$ and $x_2=0$? In other words, ...
16
votes
1answer
5k views

The difference between with or without intercept model in logistic regression

I like to understand the difference between with or without intercept model in logistic regression Is there any difference between them except that with the intercept the coefficients regard the log(...
11
votes
2answers
5k views

What does the formula y ~ x + 0 in R actually calculate?

What is the statistical difference between doing a linear regression in R with the formula set to y ~ x + 0 instead of ...
9
votes
2answers
5k views

Regression through the origin

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1) \\ (5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3) $$ How can we find the best fitting line $y=ax$ ...
8
votes
1answer
9k views

Understanding the intercept value in a multiple linear regression with categorical values

I'm failing to understand the value of the intercept value in a multiple linear regression with categorical values. Taking the "warpbreaks" data set as an example, when I do: ...
8
votes
4answers
914 views

Does the intercept in a logistic regression capture the unobserved effects?

Theoretically, does the intercept term in a logistic regression model capture all unobserved effects? In other words, in a logistic regression model with a perfect fit (i.e. all relevant variables ...
7
votes
3answers
764 views

Why is the intercept in multiple regression changing when including/excluding regressors?

I have a seemingly naive question regarding the interpretation of the intercept in multiple regression. What I found several times is something like this: The constant/intercept is defined as the ...
7
votes
1answer
597 views

Why is the intercept typed in as a 1 in stats packages (R, python)

When using statistics software, When defining your linear models, why is the intercept typed in as a 1, rather than "const" or "intercept" or something. What significance does 1 have? Is there some ...
7
votes
2answers
7k views

Deliberately fitting a model without intercept [duplicate]

Is there a situation in which the mean of a Y variable is not 0 (e.g. not standardized), but we would still fit a regression model without intercept? It would yield a worse fitting model, so is there ...
7
votes
3answers
23k views

Interpreting coefficients in a logistic regression model with a categorical variable having more than 2 levels

There is quite some content online interpreting odds in a logistic model with a dichotomous predictor. My problem is understanding coefficients when there are more than 2 levels for a categorical ...
7
votes
2answers
16k views

What exactly is the standard error of the intercept in multiple regression analysis?

I understand that in multiple regression analysis, for each independent variable, you would graph dependent variable vs independent variable and you would make a line of best fit and calculate the ...
7
votes
2answers
4k views

Bias initialization in convolutional neural network

What is the correct way to initialize biases in convolutional neural networks (tf.zeros, tf.truncated_normal, ...
7
votes
3answers
3k views

What are the uses and pitfalls of regression through the origin? [duplicate]

Spuriously high R-squared is one of the pitfalls of regression through the origin (i.e. zero-intercept models). If the predictors do not contain zeroes, then is it an extrapolation? What are the uses ...
6
votes
2answers
1k views

a regression through the origin

Why do a pair of variables with no significant correlation and no significant regression intercept and slope, have a highly significant regression with high adjusted $R^2$ when the regression is ...
6
votes
2answers
6k views

Removing intercept from GLM for multiple factorial predictors only works for first factor in model

I am running a binomial logistic regression with a logit link function in R. My response is factorial [0/1] and I have two multilevel factorial predictors - let's call them $a$ and $b$ where $a$ has 4 ...
6
votes
1answer
1k views

Why is softmax regression often written without the bias term?

I am familiar with softmax regression being written by: $$P(Y=y\mid X=x)=\frac{e^{[Wx+b]_{y}}}{\sum_{\forall i}e^{[Wx+b]_{i}}}$$ for the change of the class of $Y$ being $y$, given observations of $X$...
6
votes
1answer
3k views

Confused about 0 intercept in logistic regression in R

I'm exploring the effects of removing the intercept in a logistic regression model. Assume a model: $$logit(Y = 1) = \beta_1 x + \beta_2z + 0$$ with $x$ and $z$ being categorical variables with 2 ...
6
votes
2answers
4k views

What is the reason for not including an intercept term in AR and ARMA models?

In econometric literature it is usually argued that in case of estimating an equation, an intercept term must be always included regardless of its statistical importance because removing the constant ...
6
votes
1answer
864 views

Why and how does the inclusion of random effects in mixed models influence the fixed-effect intercept term?

The question is best illustrated by this example which uses a dataset (in library faraway) and lme4 library (both in R). This ...
5
votes
2answers
486 views

'Size' of intercept at linear regression

I have a question about this table. Why does the constant (intercept) change so dramatically from Model 1 to Model 2?
5
votes
1answer
59k views

Doing multiple regression without intercept in R (without changing data dimensions)

I am trying to calculate multiple regression in R without intercept. My data is as follow: ...
5
votes
1answer
1k views

Intercept from standardized coefficients in logistic regression

I have fit a logistic regression model with original y and standardized x variables. Slope coefficients can be easily converted back to their original scale by $\beta^*_j/\sigma_{x_j}$ where $\beta^*...
5
votes
1answer
5k views

How is the standard error of a slope calculated when the intercept term is omitted?

Let's say we have data that looks like this: ...
5
votes
1answer
4k views

impose an intercept on lm in r [duplicate]

I am converting a high-dimensional model to a lower dimensional model by fitting a sliding window of it to a linear (parametric) model and looking at the evolution of parameter values over time. I'm ...
5
votes
1answer
272 views

Logistic model output when all inputs are zero

Consider a case where I have developed a predictive model using logistic regression. Now the logistic models gives a probability even when all the inputs are zero (because of the intercept). Now ...
5
votes
1answer
4k views

How to know where to put bias terms in neural nets? [duplicate]

I've read different places that talk about bias terms in neural nets like this Importance of the bias node in neural networks But I still have trouble understanding what it is used for and how it ...
5
votes
1answer
2k views

Logistic regression: Strange standard errors from glm() in R

To my surprise I found that standard errors and thus Wald confidence intervals became smaller when I removed the intercept from a simple logistic regression model, using ...
5
votes
1answer
273 views

Is it possible to interpret regression output omitting the intercept?

Is it possible to interpret regression output when omitting the intercept? Can this omission be justified?
4
votes
1answer
12k views

Interpretation of intercept of a regression line in time series data

Does the intercept value of a regression equation have meaning in a time series dataset? Suppose I have a dataset: the intercept is 27.512, but we are 95 percent sure that the intercept is between -...
4
votes
2answers
323 views

Calculate the intercept from lm

I would like to understand how I can compute by hand the intercept from lm. The following example is a fractional factorial design (3^3) and the variables are factors. ...
4
votes
2answers
9k views

Logistic regression intercept representing baseline probability

In a linear regression, when you standardize your numeric variables, the resulting intercept has the same value as the mean of your sample. Is there any way in a logistic regression, with numeric ...
4
votes
1answer
346 views

Significance of intercept (as portrayed via an R formula)

I'm new to statistics in general (but a very seasoned developer). I'm trying to grasp why it seems like there's a lot of consideration given to intercepts, at least where it comes to models. For ...
4
votes
1answer
1k views

Is it ok to remove the intercept in a linear regression model (OLS) if the results are really good? [duplicate]

So I've gone through this SE question and all the answers where the general consensus is that you should never remove the intercept of the linear regression model. The most upvoted answer says: The ...
4
votes
1answer
245 views

Zero-intercept poisson regression model predicts better than a model with an intercept?

I have read some blogs/articles saying that intercept should not be suppressed. Recently, I used glmm.admb to model a ZIP (Zero-inflated Poisson) model which is giving better results without an ...
4
votes
1answer
107 views

Detect a change in linear regression model's error

At time T0: A linear model is deployed to predict outcome Y as a function of X with equation Y = m1*x + c1 At time T1: The underlying process drifts to a new mean, resulting in consistently positive ...
4
votes
1answer
2k views

Regression coefficients without intercept [duplicate]

Could someone recommend a link or help me out here: where can I find the formula for the regression without an intercept, and how is it deriveed differently than the formula with the intercept? (...
4
votes
1answer
908 views

GMM estimation of linear regression with intercept restriction

Say I have a time series regression as follows: $$y_t = a_i + \beta_i x_t + \varepsilon_t^i \ \ ; \ \ t = 1, 2, \cdots, T \ \ \text{for each } i$$ Now say I impose the following restriction on the ...
4
votes
0answers
523 views

Simulate event rate in logistic regression - Finding the intercept

I want to simulate a logistic regression (using a set of continous and binary confounders with known odds ratios) with a specified probability outcome (e.g. event rate = 0.2). This is actually a ...
3
votes
4answers
2k views

Linear regression with negative estimated value for intercept

Does a negative value of intercept suggest that the regression line provides poor fit to the data? why? and why not?
3
votes
2answers
288 views

nonsensical intercepts for regression models

Let’s say that I have performances in 9 sports as predictor variables and the total points of those sports as the DV. So I am making three regression models(non-nested) with three predictors each (...
3
votes
3answers
3k views

How does the inclusion of an intercept change the variability of the residual?

I want to use the variability of the residual as a measure M and then test whether M is higher or lower after some event. However, I estimate separate regression before and after the event to obtain ...

1
2 3 4 5 6