Questions tagged [intercept]

The intercept in regression-type models is the value of the Y variable when all X variables are 0.

Filter by
Sorted by
Tagged with
155
votes
9answers
171k views

When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the ...
126
votes
2answers
67k views

Removal of statistically significant intercept term increases $R^2$ in linear model

In a simple linear model with a single explanatory variable, $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$ I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
25
votes
8answers
68k views

When forcing intercept of 0 in linear regression is acceptable/advisable [duplicate]

I have a regression model to estimate the completion time of a process, based on various factors. I have 200 trials of these processes, where the 9 factors being measured vary widely. When I perform a ...
6
votes
2answers
8k views

Removing intercept from GLM for multiple factorial predictors only works for first factor in model

I am running a binomial logistic regression with a logit link function in R. My response is factorial [0/1] and I have two multilevel factorial predictors - let's call them $a$ and $b$ where $a$ has 4 ...
17
votes
3answers
66k views

Intercept term in logistic regression

Suppose we have the following logistic regression model: $$\text{logit}(p) = \beta_0+\beta_{1}x_{1} + \beta_{2}x_{2}$$ Is $\beta_0$ the odds of the event when $x_1 = 0$ and $x_2=0$? In other words, ...
16
votes
1answer
6k views

The difference between with or without intercept model in logistic regression

I like to understand the difference between with or without intercept model in logistic regression Is there any difference between them except that with the intercept the coefficients regard the log(...
7
votes
3answers
3k views

Why is the intercept in multiple regression changing when including/excluding regressors?

I have a seemingly naive question regarding the interpretation of the intercept in multiple regression. What I found several times is something like this: The constant/intercept is defined as the ...
9
votes
2answers
6k views

Regression through the origin

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1) \\ (5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3) $$ How can we find the best fitting line $y=ax$ ...
22
votes
6answers
14k views

Reason for not shrinking the bias (intercept) term in regression

For a linear model $y=\beta_0+x\beta+\varepsilon$, the shrinkage term is always $P(\beta) $. What is the reason that we do not shrink the bias (intercept) term $\beta_0$? Should we shrink the bias ...
2
votes
1answer
128 views

Both variables of my GLMM output are significant. Don't know how to interpret it?

This is more of an interpretation question than anything. I have run a GLMM with two fixed factors (both of which have two levels) and two random factors. The outputs from the model are as such: <...
2
votes
1answer
4k views

Ways of comparing linear regression intercepts and slopes?

I'm a little bit confused about this, so any help would be appreciated! Let's say I have a repeated-measures design in which participants take part in a task where they have to rate the ...
23
votes
3answers
16k views

Importance of the bias node in neural networks

I'm curious to know how important the bias node is for the effectiveness of modern neural networks. I can easily understand that it can be important in a shallow network with only a few input ...
20
votes
3answers
10k views

Why would one suppress the intercept in linear regression?

In a number of statistical packages including SAS, SPSS and maybe more, there is an option to "suppress the intercept". Why would you want to do that?
2
votes
1answer
3k views

Use squared correlation in regression without intercept [duplicate]

If I want to compare the goodness-of-fit of two regression models, with and without intercept, is it valid to compare the squared correlation coefficient between the fitted values and the data? Since ...
1
vote
1answer
3k views

Should I keep a non-significant intercept in a GARCH model?

I've estimated an ARMA-GARCH model. All the parameters except the intercept ($\omega$) in the GARCH part are significant. So I tried to estimate the model again imposing that $\omega = 0$. I compared ...
7
votes
2answers
8k views

Deliberately fitting a model without intercept [duplicate]

Is there a situation in which the mean of a Y variable is not 0 (e.g. not standardized), but we would still fit a regression model without intercept? It would yield a worse fitting model, so is there ...
7
votes
3answers
3k views

What are the uses and pitfalls of regression through the origin? [duplicate]

Spuriously high R-squared is one of the pitfalls of regression through the origin (i.e. zero-intercept models). If the predictors do not contain zeroes, then is it an extrapolation? What are the uses ...
32
votes
3answers
55k views

Why are bias nodes used in neural networks?

Why are bias nodes used in neural networks? How many you should use? In which layers you should use them: all hidden layers and the output layer?
24
votes
1answer
11k views

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them ...
3
votes
2answers
7k views

Convert 'intercept/drift' term to 'constant' term in arima/auto.arima functions, in case of higher orders - R

suppose for a time series, the auto.arima output is: ...
5
votes
2answers
12k views

Logistic regression intercept representing baseline probability

In a linear regression, when you standardize your numeric variables, the resulting intercept has the same value as the mean of your sample. Is there any way in a logistic regression, with numeric ...
7
votes
2answers
6k views

What is the reason for not including an intercept term in AR and ARMA models?

In econometric literature it is usually argued that in case of estimating an equation, an intercept term must be always included regardless of its statistical importance because removing the constant ...
6
votes
1answer
3k views

Confused about 0 intercept in logistic regression in R

I'm exploring the effects of removing the intercept in a logistic regression model. Assume a model: $$logit(Y = 1) = \beta_1 x + \beta_2z + 0$$ with $x$ and $z$ being categorical variables with 2 ...
5
votes
1answer
291 views

Is it possible to interpret regression output omitting the intercept?

Is it possible to interpret regression output when omitting the intercept? Can this omission be justified?
6
votes
2answers
2k views

a regression through the origin

Why do a pair of variables with no significant correlation and no significant regression intercept and slope, have a highly significant regression with high adjusted $R^2$ when the regression is ...
1
vote
1answer
97 views

Computation of the intercept in logistic regression model

I'm trying to understand the way the odds of the reference groups are computed. Let's consider an example from this paper. Data can be summarised in the table: The reference group is Older and New. ...
9
votes
1answer
10k views

Understanding the intercept value in a multiple linear regression with categorical values

I'm failing to understand the value of the intercept value in a multiple linear regression with categorical values. Taking the "warpbreaks" data set as an example, when I do: ...
11
votes
2answers
6k views

What does the formula y ~ x + 0 in R actually calculate?

What is the statistical difference between doing a linear regression in R with the formula set to y ~ x + 0 instead of ...
3
votes
1answer
2k views

VAR models in R: Constant, Trend, Constant+Trend, None

I currently apply a VAR models with the vars package in R. There are the options to select "both", "none", ...
13
votes
5answers
251 views

Different usage of the term "Bias" in stats/machine learning

I think I've seen about 4 different usages of the word "bias" in stats/ML, and all these usages seem to be non-related. I just wanted to get clarification that the usages are indeed non-...
4
votes
1answer
405 views

Significance of intercept (as portrayed via an R formula)

I'm new to statistics in general (but a very seasoned developer). I'm trying to grasp why it seems like there's a lot of consideration given to intercepts, at least where it comes to models. For ...
7
votes
2answers
23k views

What exactly is the standard error of the intercept in multiple regression analysis?

I understand that in multiple regression analysis, for each independent variable, you would graph dependent variable vs independent variable and you would make a line of best fit and calculate the ...
3
votes
1answer
2k views

Fitting simple linear regression with no intercept

Say we fit a simple linear regression without the intercept term. I know this is inadvisable for the most part, unless there are some situations where it is reasonable to assume that when $x=0$, $y$ ...
6
votes
1answer
1k views

Intercept from standardized coefficients in logistic regression

I have fit a logistic regression model with original y and standardized x variables. Slope coefficients can be easily converted back to their original scale by $\beta^*_j/\sigma_{x_j}$ where $\beta^*...
3
votes
2answers
9k views

Role of the bias term in regression [closed]

I was trying to understand the role of the bias term in linear regression which is given by, y=w^T. phi(x)+b From what I understand it allows for any fixed ...
2
votes
0answers
217 views

Are HAC robust standard errors robust against autoregressive conditional heteroskedasticity?

Suppose I have a GARCH(p,q) model with constant conditional mean, \begin{aligned} y_t &= \mu + u_t, \\ u_t &= \sigma_t \varepsilon_t, \\ \sigma_t^2 &= \omega + \alpha_1 u_{t-1}^2 + \dotsc +...
7
votes
1answer
2k views

Why is softmax regression often written without the bias term?

I am familiar with softmax regression being written by: $$P(Y=y\mid X=x)=\frac{e^{[Wx+b]_{y}}}{\sum_{\forall i}e^{[Wx+b]_{i}}}$$ for the change of the class of $Y$ being $y$, given observations of $X$...
2
votes
1answer
134 views

If I consider the fixed factor as a random slope, the p-value changes from p<0,05 to p>0,05

I'm having a hard time trying to understand the differences between these two models and why the first one shows correlation (p-value < 0,05) but the other one doesn´t (p-value > 0,05). I would ...
2
votes
0answers
53 views

Is it valid to solve an equation for multiple coefficients, then average them to obtain overall effect?

I have a regression model, the setup for which is as follows: I am using manyglm, a multivariate general linear model approach to determine the difference in several invertebrate species between two ...
0
votes
1answer
1k views

Which intercept R selects (binomial glm)?

I have a problem with an analysis. I'm doing a binomial glm with two categorical factors that are loc and trat. I do not understand how R deals with the intercept (what statistical explanation does ...
6
votes
1answer
966 views

Why and how does the inclusion of random effects in mixed models influence the fixed-effect intercept term?

The question is best illustrated by this example which uses a dataset (in library faraway) and lme4 library (both in R). This ...
5
votes
1answer
3k views

Logistic regression: Strange standard errors from glm() in R

To my surprise I found that standard errors and thus Wald confidence intervals became smaller when I removed the intercept from a simple logistic regression model, using ...
4
votes
1answer
503 views

Calculating variance components and ICC of a random intercept model by hand

When I really want to understand a measure or parameter, I tend to do the calculation by hand with simplified data. Today I have attempted to do the same with the ICC, but somehow keep failing. I was ...
4
votes
1answer
14k views

Interpretation of intercept of a regression line in time series data

Does the intercept value of a regression equation have meaning in a time series dataset? Suppose I have a dataset: the intercept is 27.512, but we are 95 percent sure that the intercept is between -...
3
votes
1answer
99 views

Interpreting the intercept in logistic regression with a binary variable

I am using R to run a logistic regression to analyze how a categorical variable ("population") correlates with a binary variable ("response") and am having some trouble ...
3
votes
1answer
64 views

Interpreting estimates of a bivariate regression model with a categotical and a numeric variable

How to interpret the intercept in a bivariate regression with one numeric and one categorical variable? The following model has a numeric variable (log10(N_Total_e)), namely the log transformed ...
3
votes
1answer
809 views

Does R glmnet regularize on intercept?

Both this post and original paper, suggest R glmnet does not regularize the intercept. But why I am observing the intercept ...
2
votes
1answer
742 views

Newey West standard errors in regression model without constant

I'm estimating $y_i= \beta_1 \times x_{1i} + \varepsilon_i$ on a time series on $y$ and $x$, so in presence of heteroskedasticity and autocorrelation. My model does not include any intercepts. Are ...
2
votes
1answer
960 views

I computed ARMA equation from R manually but never got the same result with predict() or forecast() provided by R

I've got a little problem here. I've been doing analysis with time series data using ARMA, and it always turns out that the parameters I get from R didn't fit to my computation when I do it manually. ...