Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression-coefficients]

The parameters of a regression model. Most commonly, the values by which the independent variables will be multiplied to get the predicted value of the dependent variable.

83
votes
17answers
66k views

Including the interaction but not the main effects in a model

Is it ever valid to include a two-way interaction in a model without including the main effects? What if your hypothesis is only about the interaction, do you still need to include the main effects?
61
votes
1answer
65k views

How to interpret coefficients in a Poisson regression?

How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression? Assume the following example: ...
42
votes
3answers
45k views

Interpretation of log transformed predictor and/or response

I'm wondering if it makes a difference in interpretation whether only the dependent, both the dependent and independent, or only the independent variables are log transformed. Consider the case of <...
35
votes
3answers
28k views

How to interpret coefficients from a polynomial model fit?

I'm trying to create a second order polynomial fit to some data I have. Let's say I plot this fit with ggplot(): ...
34
votes
2answers
40k views

Multiple regression or partial correlation coefficient? And relations between the two

I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
32
votes
3answers
60k views

Derive Variance of regression coefficient in simple linear regression

In simple linear regression, we have $y = \beta_0 + \beta_1 x + u$, where $u \sim iid\;\mathcal N(0,\sigma^2)$. I derived the estimator: $$ \hat{\beta_1} = \frac{\sum_i (x_i - \bar{x})(y_i - \bar{y})}...
24
votes
3answers
8k views

Does the order of explanatory variables matter when calculating their regression coefficients?

At first I thought the order didn’t matter, but then I read about the gram-schmidt orthogonalization process for calculating multiple regression coefficients, and now I’m having second thoughts. ...
23
votes
1answer
16k views

Is there a way to use the covariance matrix to find coefficients for multiple regression?

For simple linear regression, the regression coefficient is calculable directly from the variance-covariance matrix $C$, by $$ C_{d, e}\over C_{e,e} $$ where $d$ is the dependent variable's index, ...
22
votes
2answers
15k views

How does bootstrapping in R actually work?

I've been looking into the boot package in R and while I have found a number of good primers on how to use it, I have yet to find anything that describes exactly what is happening "behind the scenes". ...
22
votes
3answers
12k views

What does “all else equal” mean in multiple regression?

When we do multiple regressions and say we are looking at the average change in the $y$ variable for a change in an $x$ variable, holding all other variables constant, what values are we holding the ...
20
votes
1answer
11k views

interpreting estimates of cloglog logistic regression

Could someone advise me on how to interpret the estimates from a logistic regression using a cloglog link? I have fitted the following model in lme4: ...
20
votes
4answers
16k views

Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients

I am wondering what the exact relationship between partial $R^2$ and coefficients in a linear model is and whether I should use only one or both to illustrate the importance and influence of factors. ...
18
votes
3answers
18k views

How to compute the standard errors of a logistic regression's coefficients

I am using Python's scikit-learn to train and test a logistic regression. scikit-learn returns the regression's coefficients of the independent variables, but it does not provide the coefficients' ...
17
votes
3answers
47k views

How to interpret main effects when the interaction effect is not significant?

I ran a Generalized Linear Mixed Model in R and included an interaction effect between two predictors. The interaction was not significant, but the main effects (the two predictors) both were. Now ...
17
votes
2answers
10k views

Calculate coefficients in a logistic regression with R

In a multiple linear regression it is possible to find out the coeffient with the following formula. $b = (X'X)^{-1}(X')Y$ ...
16
votes
1answer
7k views

Question on how to normalize regression coefficient

Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares. Suppose you have $y=\beta_0+\beta_1x_1$, ...
15
votes
1answer
18k views

How to interpret the coefficients from a beta regression?

I have some data that is bounded between 0 and 1. I have used the betareg package in R to fit a regression model with the bounded data as the dependent variable. My ...
15
votes
2answers
17k views

Interpretation of betas when there are multiple categorical variables

I understand the concept that $\hat\beta_0$ is the mean for when the categorical variable is equal to 0 (or is the reference group), giving the end interpretation that the regression coefficient is ...
15
votes
1answer
45k views

Standard errors for multiple regression coefficients?

I realize that this is a very basic question, but I can't find an answer anywhere. I'm computing regression coefficients using either the normal equations or QR decomposition. How can I compute ...
15
votes
1answer
3k views

How to treat categorical predictors in LASSO

I am running a LASSO that has some categorical variable predictors and some continuous ones. I have a question about the categorical variables. The first step I understand is to break each of them ...
15
votes
1answer
395 views

An unbiased estimator of the ratio of two regression coefficients?

Suppose you fit a linear/logistic regression $g(y) = a_0 + a_1\cdot x_1 + a_2\cdot x_2$, with the aim of an unbiased estimate of $\frac{a_1}{a_2}$. You are very confident that both $a_1$ and $a_2$ ...
14
votes
2answers
576 views

Sparsity by discarding least squares' coefficient

Suppose I wish to regress $Y$ against a normalized $X$, but I would like a sparse solution. After regression, why is discarding the coefficients with smallest magnitude not allowed? For the record, ...
14
votes
2answers
1k views

Regression to the mean in “Thinking, Fast and Slow”

In Thinking, Fast and Slow, Daniel Kahneman poses the following hypothetical question: (P. 186) Julie is currently a senior in a state university. She read fluently when she was four years old. ...
14
votes
1answer
3k views

Ridge regression coefficients that are larger than OLS coefficients or that change sign depending on $\lambda$

When running ridge regression, how do you interpret coefficients that end up larger than their corresponding coefficients under least squares (for certain values of $\lambda$)? Isn't ridge regression ...
13
votes
2answers
58k views

How to deal with an error such as “Coefficients: 14 not defined because of singularities” in R?

When doing a GLM and you get the "not defined because of singularities" error in the anova output, how does one counteract this error from happening? Some have suggested that it is due to ...
13
votes
3answers
17k views

Do coefficients of logistic regression have a meaning?

I have a binary classification problem from several features. Do the coefficients of a (regularized) logistic regression have an interpretable meaning? I thought they could indicate the size of ...
13
votes
1answer
17k views

What's the difference between regression coefficients and partial regression coefficients?

I've read in Abdi (2003) that When the independent variables are pairwise orthogonal, the effect of each of them in the regression is assessed by computing the slope of the regression between ...
13
votes
1answer
2k views

Recovering raw coefficients and variances from orthogonal polynomial regression

It seems that if I have a regression model such as $y_i \sim \beta_0 + \beta_1 x_i+\beta_2 x_i^2 +\beta_3 x_i^3$ I can either fit a raw polynomial and get unreliable results or fit an orthogonal ...
12
votes
5answers
9k views

Can I ignore coefficients for non-significant levels of factors in a linear model?

After seeking clarification about linear model coefficients over here I have a follow up question concerning non-signficant (high p value) for coefficients of factor levels. Example: If my linear ...
12
votes
6answers
35k views

Acceptable r-square value for multiple linear regression model

I'm currently working on my thesis, more specifically I'm analyzing some data collected from researchers about the project's they're working on. In the end, I have performed a multiple linear ...
12
votes
2answers
21k views

Positive correlation and negative regressor coefficient sign

Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression ...
12
votes
4answers
19k views

“Moderation” versus “interaction”?

I have come across these two terms which are used interchangeably in many contexts. Basically, a moderator (M) is a factor that impacts on the relationship between X and Y. Moderation analysis is ...
12
votes
2answers
25k views

Interpretation of incidence-rate ratios

So, I want to fit a random effects negativ-binomial model. For such a model STATA can produce exponentiated coefficients. According to the help file such coefficients can be interpreted as incidence-...
12
votes
2answers
15k views

What does r, r squared and residual standard deviation tell us about a linear relationship?

Little background I'm working on the interpretation of regression analysis but I get really confused about the meaning of r, r squared and residual standard deviation. I know the definitions: ...
11
votes
3answers
1k views

Why does the product of the bivariate regression coefficients of the $y$-on-$x$ line and $x$-on-$y$ line equal the square of the correlation?

There's regression model where $Y = a + bX$ with $a = 1.6$ and $b=0.4$, which has a correlation coefficient of $r = 0.60302$. If $X$ and $Y$ are then switched around and the equation becomes $X = c + ...
11
votes
2answers
15k views

How to convert standardized coefficients to unstandardized coefficients?

My goal is to use the coefficients derived by previous research on the subject to predict actual outcomes given a set of independent variables. However, the research paper lists the Beta coefficients ...
11
votes
4answers
15k views

Standardized beta weights for a multilevel regression

How can one obtain standardized (fixed effect) regression weights from a multilevel regression? And, as an "add-on": What is the easiest way to obtain these standardized weights from a ...
11
votes
1answer
11k views

Interpretation of LASSO regression coefficients

I'm currently working on building a predictive model for a binary outcome on a dataset with ~300 variables and 800 observations. I've read much on this site about the problems associated with stepwise ...
11
votes
1answer
9k views

How to interpret a negative linear regression coefficient for a logged outcome variable?

I have a linear regression model where the dependent variable is logged and an independent variable is linear. The slope coefficient for a key independent variable is negative: $-.0564$. Not sure how ...
11
votes
1answer
2k views

Comparing regression coefficients of same model across different data sets

I'm evaluating two (2) refrigerants (gases) that were used in the same refrigeration system. I have saturated suction temperature ($S$), condensing temperature ($D$), and amperage ($Y$) data for the ...
10
votes
4answers
18k views

How to interpret logarithmically transformed coefficients in linear regression?

My situation is: I have 1 continuous dependent and 1 continuous predictor variable that I've logarithmically transformed to normalise their residuals for simple linear regression. I would ...
10
votes
3answers
9k views

Joint model with interaction terms vs. separate regressions for a group comparison

After gathering valuable feedback from previous questions and discussions, I have came up with the following question: Suppose that the goal is to detect effect differences across two groups, male vs. ...
10
votes
1answer
10k views

How to compare two regression slopes for one predictor on two different outcomes?

I need to compare two regression slopes where: $ y_1 ~ a + b_1x y_2 ~ a + b_2x $ How can I compare b1 and b2? Or in the language of my specific example in ...
10
votes
1answer
8k views

Difference between marginal and conditional models

A marginal model accounts for the correlation within each cluster. A conditional model also takes into account the correlation within each cluster. My questions are: Does a marginal model models ...
9
votes
3answers
13k views

Is it possible in R (or in general) to force regression coefficients to be a certain sign?

I'm working with some real world data and the regression models are yielding some counterintuitive results. Normally I trust the statistics but in reality some of these things can not be true. The ...
9
votes
2answers
3k views

Compare the statistical significance of the difference between two polynomial regressions in R

So first of all I did some research on this forum, and I know extremely similar questions have been asked but they usually haven't been answered properly or sometimes the answer are simply not ...
9
votes
3answers
287 views

What is this bias-variance tradeoff for regression coefficients and how to derive it?

In this paper, (Bayesian Inference for Variance Components Using Only Error Contrasts, Harville, 1974), the author claims $$(y-X\beta)'H^{-1}(y-X\beta)=(y-X\hat\beta)'H^{-1}(y-X\hat\beta)+(\beta-\hat\...
9
votes
1answer
7k views

R linear regression categorical variable “hidden” value

This is just an example that I have come across several times, so I don't have any sample data. Running a linear regression model in R: a.lm = lm(Y ~ x1 + x2) <...
9
votes
2answers
7k views

Testing for coefficients significance in Lasso logistic regression

[A similar question was asked here with no answers] I have fit a logistic regression model with L1 regularization (Lasso logistic regression) and I would like to test the fitted coefficients for ...
9
votes
1answer
10k views

Cox proportional hazard model and interpretation of coefficients when higher case interaction is involved

Here is the summary-output of the Coxph-model I used (I used R and the output is based on the best final model i.e. all significant explanatory variables and their interactions are included): ...