Questions tagged [regression-coefficients]

The parameters of a regression model. Most commonly, the values by which the independent variables will be multiplied to get the predicted value of the dependent variable.

Filter by
Sorted by
Tagged with
43
votes
3answers
47k views

Interpretation of log transformed predictor and/or response

I'm wondering if it makes a difference in interpretation whether only the dependent, both the dependent and independent, or only the independent variables are log transformed. Consider the case of <...
84
votes
17answers
67k views

Including the interaction but not the main effects in a model

Is it ever valid to include a two-way interaction in a model without including the main effects? What if your hypothesis is only about the interaction, do you still need to include the main effects?
16
votes
1answer
7k views

Question on how to normalize regression coefficient

Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares. Suppose you have $y=\beta_0+\beta_1x_1$, ...
23
votes
1answer
17k views

Is there a way to use the covariance matrix to find coefficients for multiple regression?

For simple linear regression, the regression coefficient is calculable directly from the variance-covariance matrix $C$, by $$ C_{d, e}\over C_{e,e} $$ where $d$ is the dependent variable's index, ...
13
votes
5answers
9k views

Can I ignore coefficients for non-significant levels of factors in a linear model?

After seeking clarification about linear model coefficients over here I have a follow up question concerning non-signficant (high p value) for coefficients of factor levels. Example: If my linear ...
34
votes
2answers
41k views

Multiple regression or partial correlation coefficient? And relations between the two

I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
6
votes
5answers
10k views

Coefficient changes sign when adding a variable in logistic regression

In my logistic regression the sign of coefficients of a variable (location distance of an amenity) changes based on other variables (with time -ve, with travel distance +ve) in the model. When the ...
61
votes
1answer
67k views

How to interpret coefficients in a Poisson regression?

How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression? Assume the following example: ...
7
votes
1answer
5k views

Interpretation of interaction term

I have a model: $$ \ln({\rm earnings}) = a+b_1{\rm female}+b_2{\rm white}+b_3{\rm female}\times{\rm white} $$ ${\rm female}$ and ${\rm white}$ are dummy variables. I have interpreted $b_1$ and $b_2$: ...
22
votes
3answers
12k views

What does “all else equal” mean in multiple regression?

When we do multiple regressions and say we are looking at the average change in the $y$ variable for a change in an $x$ variable, holding all other variables constant, what values are we holding the ...
36
votes
3answers
62k views

Derive Variance of regression coefficient in simple linear regression

In simple linear regression, we have $y = \beta_0 + \beta_1 x + u$, where $u \sim iid\;\mathcal N(0,\sigma^2)$. I derived the estimator: $$ \hat{\beta_1} = \frac{\sum_i (x_i - \bar{x})(y_i - \bar{y})}...
18
votes
2answers
18k views

Interpretation of betas when there are multiple categorical variables

I understand the concept that $\hat\beta_0$ is the mean for when the categorical variable is equal to 0 (or is the reference group), giving the end interpretation that the regression coefficient is ...
12
votes
2answers
22k views

Positive correlation and negative regressor coefficient sign

Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression ...
20
votes
4answers
16k views

Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients

I am wondering what the exact relationship between partial $R^2$ and coefficients in a linear model is and whether I should use only one or both to illustrate the importance and influence of factors. ...
36
votes
4answers
30k views

How to interpret coefficients from a polynomial model fit?

I'm trying to create a second order polynomial fit to some data I have. Let's say I plot this fit with ggplot(): ...
8
votes
3answers
7k views

How to apply coefficient term for factors and interactive terms in a linear equation?

Using R, I have fitted a linear model for a single response variable from a mix of continuous and discrete predictors. This is uber-basic, but I'm having trouble grasping how a coefficient for a ...
7
votes
3answers
2k views

Explicit solution for linear regression with two predictors

I have some samples of data of the form $x,y$ and $z=f(x,y)$. I wish to fit a plane $$z = Ax + By + C$$ to the data with the smallest mean square errors. I have found an "answer" in section 3 of this ...
9
votes
2answers
3k views

Compare the statistical significance of the difference between two polynomial regressions in R

So first of all I did some research on this forum, and I know extremely similar questions have been asked but they usually haven't been answered properly or sometimes the answer are simply not ...
10
votes
4answers
18k views

How to interpret logarithmically transformed coefficients in linear regression?

My situation is: I have 1 continuous dependent and 1 continuous predictor variable that I've logarithmically transformed to normalise their residuals for simple linear regression. I would ...
14
votes
1answer
2k views

Recovering raw coefficients and variances from orthogonal polynomial regression

It seems that if I have a regression model such as $y_i \sim \beta_0 + \beta_1 x_i+\beta_2 x_i^2 +\beta_3 x_i^3$ I can either fit a raw polynomial and get unreliable results or fit an orthogonal ...
10
votes
1answer
10k views

How to compare two regression slopes for one predictor on two different outcomes?

I need to compare two regression slopes where: $ y_1 ~ a + b_1x y_2 ~ a + b_2x $ How can I compare b1 and b2? Or in the language of my specific example in ...
24
votes
3answers
8k views

Does the order of explanatory variables matter when calculating their regression coefficients?

At first I thought the order didn’t matter, but then I read about the gram-schmidt orthogonalization process for calculating multiple regression coefficients, and now I’m having second thoughts. ...
14
votes
4answers
19k views

“Moderation” versus “interaction”?

I have come across these two terms which are used interchangeably in many contexts. Basically, a moderator (M) is a factor that impacts on the relationship between X and Y. Moderation analysis is ...
11
votes
2answers
15k views

How to convert standardized coefficients to unstandardized coefficients?

My goal is to use the coefficients derived by previous research on the subject to predict actual outcomes given a set of independent variables. However, the research paper lists the Beta coefficients ...
6
votes
2answers
4k views

Linear regression with log transformed data - large error [duplicate]

I have a set of data which is has a very large positive skew, and has been transformed using a logarithm. I wish to predict one variable from another using the lm ...
4
votes
2answers
2k views

Efficiency of beta estimates with heteroscedasticity

I need something clarified and that is when you have non-constant variance, estimates won't be biased but will be a problem when it comes to the S.E. formulas and efficiency. Therefore OLS estimates ...
4
votes
2answers
2k views

Is the model wrong if a coefficient changes from minus in correlation table to plus in OLS?

Perhaps a very basic question but one that has me confused. Say, in a correlation table the relationship between A and the DV (B)...
2
votes
1answer
4k views

R: linear regression: very small coefficient and R-squared but significant P values

I've got a very small coefficient (-0.04) and R-squared (0.028) but a significant P value (<0.0001). My question is: Is my result still meaningful? How to interpret it? The result is from a ...
17
votes
2answers
11k views

Calculate coefficients in a logistic regression with R

In a multiple linear regression it is possible to find out the coeffient with the following formula. $b = (X'X)^{-1}(X')Y$ ...
18
votes
3answers
48k views

How to interpret main effects when the interaction effect is not significant?

I ran a Generalized Linear Mixed Model in R and included an interaction effect between two predictors. The interaction was not significant, but the main effects (the two predictors) both were. Now ...
11
votes
3answers
9k views

Joint model with interaction terms vs. separate regressions for a group comparison

After gathering valuable feedback from previous questions and discussions, I have came up with the following question: Suppose that the goal is to detect effect differences across two groups, male vs. ...
8
votes
2answers
4k views

How can $R^2$ have two different values for the same regression (without an intercept) [duplicate]

My question is multifaceted. So I'll start by asking my question and then explain what has caused me to ask this question. How can I calculate the coefficient of determination for a linear ...
11
votes
1answer
12k views

Interpretation of LASSO regression coefficients

I'm currently working on building a predictive model for a binary outcome on a dataset with ~300 variables and 800 observations. I've read much on this site about the problems associated with stepwise ...
10
votes
1answer
9k views

Difference between marginal and conditional models

A marginal model accounts for the correlation within each cluster. A conditional model also takes into account the correlation within each cluster. My questions are: Does a marginal model models ...
11
votes
3answers
1k views

Why does the product of the bivariate regression coefficients of the $y$-on-$x$ line and $x$-on-$y$ line equal the square of the correlation?

There's regression model where $Y = a + bX$ with $a = 1.6$ and $b=0.4$, which has a correlation coefficient of $r = 0.60302$. If $X$ and $Y$ are then switched around and the equation becomes $X = c + ...
8
votes
3answers
16k views

Interpreting coefficient in a linear regression model with categorical variables

I will give my examples with R calls. First a simple example of a linear regression with a dependent variable 'lifespan', and two continuous explanatory variables. ...
7
votes
1answer
2k views

Significance of individual coefficients vs Significance of both

This was a question I read from google quantitative analyst interview on glassdoor: If each of the two coefficient estimates in a regression model is statistically significant, do you expect the test ...
7
votes
1answer
3k views

How to interpret log-log regression coefficients for other than 1 or 10 percent change?

I have read many threads here on how to interpret coefficients in a regression where the predictor and the dependent variable are log-transformed. Most give an answer for a one or ten percent change. ...
5
votes
2answers
905 views

Distribution of linear combination of OLS regression coefficients

I have a simple linear OLS regression $Y_i = \alpha+ \beta_1 X_{1i} + \beta_2 X_{2i} + e_i$ where $e_i \sim N(0,\sigma)$. I have estimated the regression from the data and obtained estimates for my ...
1
vote
0answers
65 views

How to derive correlation using regression without empirical proof?

I just finished learning MLE, Regression, Covariance and now in to Correlation.I want to transform logically from Regression to Correlation using Covariance. Regression: A simple regression model ...
22
votes
2answers
15k views

How does bootstrapping in R actually work?

I've been looking into the boot package in R and while I have found a number of good primers on how to use it, I have yet to find anything that describes exactly what is happening "behind the scenes". ...
15
votes
1answer
18k views

How to interpret the coefficients from a beta regression?

I have some data that is bounded between 0 and 1. I have used the betareg package in R to fit a regression model with the bounded data as the dependent variable. My ...
16
votes
1answer
46k views

Standard errors for multiple regression coefficients?

I realize that this is a very basic question, but I can't find an answer anywhere. I'm computing regression coefficients using either the normal equations or QR decomposition. How can I compute ...
14
votes
2answers
61k views

How to deal with an error such as “Coefficients: 14 not defined because of singularities” in R?

When doing a GLM and you get the "not defined because of singularities" error in the anova output, how does one counteract this error from happening? Some have suggested that it is due to ...
8
votes
4answers
29k views

How do I reference a regression model's coefficient's standard errors? [closed]

Estimate Std. Error t value Pr(>|t|) (Intercept) 10.2758 0.5185 19.817 < 2e-16 *** rprice2 -1.8581 0.5139 -3.616 0.000696 *** ...
12
votes
6answers
35k views

Acceptable r-square value for multiple linear regression model [duplicate]

I'm currently working on my thesis, more specifically I'm analyzing some data collected from researchers about the project's they're working on. In the end, I have performed a multiple linear ...
11
votes
4answers
15k views

Standardized beta weights for a multilevel regression

How can one obtain standardized (fixed effect) regression weights from a multilevel regression? And, as an "add-on": What is the easiest way to obtain these standardized weights from a ...
9
votes
1answer
7k views

R linear regression categorical variable “hidden” value

This is just an example that I have come across several times, so I don't have any sample data. Running a linear regression model in R: a.lm = lm(Y ~ x1 + x2) <...
11
votes
1answer
9k views

How to interpret a negative linear regression coefficient for a logged outcome variable?

I have a linear regression model where the dependent variable is logged and an independent variable is linear. The slope coefficient for a key independent variable is negative: $-.0564$. Not sure how ...
9
votes
2answers
8k views

Testing for coefficients significance in Lasso logistic regression

[A similar question was asked here with no answers] I have fit a logistic regression model with L1 regularization (Lasso logistic regression) and I would like to test the fitted coefficients for ...