Questions tagged [regression-coefficients]

The parameters of a regression model. Most commonly, the values by which the independent variables will be multiplied to get the predicted value of the dependent variable.

Filter by
Sorted by
Tagged with
9
votes
2answers
8k views

Testing for coefficients significance in Lasso logistic regression

[A similar question was asked here with no answers] I have fit a logistic regression model with L1 regularization (Lasso logistic regression) and I would like to test the fitted coefficients for ...
9
votes
1answer
10k views

Cox proportional hazard model and interpretation of coefficients when higher case interaction is involved

Here is the summary-output of the Coxph-model I used (I used R and the output is based on the best final model i.e. all significant explanatory variables and their interactions are included): ...
9
votes
1answer
227 views

High dimensional, correlated data and top features/ covariates discovered; multiple hypothesis testing?

I have a dataset with about 5,000 often correlated features / covariates and a binary response. The data was given to me, I didn't collect it. I use Lasso and gradient boosting to build models. I ...
8
votes
3answers
16k views

Interpreting coefficient in a linear regression model with categorical variables

I will give my examples with R calls. First a simple example of a linear regression with a dependent variable 'lifespan', and two continuous explanatory variables. ...
8
votes
2answers
2k views

Interpreting logistic regression coefficients with a regularization term

I understand the coefficients of a logistic equation can be interpreted as odd ratio. If a regularization term is added to control for over-fitting, how does this change the interpretation of the ...
8
votes
4answers
29k views

How do I reference a regression model's coefficient's standard errors? [closed]

Estimate Std. Error t value Pr(>|t|) (Intercept) 10.2758 0.5185 19.817 < 2e-16 *** rprice2 -1.8581 0.5139 -3.616 0.000696 *** ...
8
votes
2answers
4k views

How can $R^2$ have two different values for the same regression (without an intercept) [duplicate]

My question is multifaceted. So I'll start by asking my question and then explain what has caused me to ask this question. How can I calculate the coefficient of determination for a linear ...
8
votes
2answers
4k views

Regression: What is the utility of R squared compared to RMSE?

Suppose I'm doing regression with training, validation, and test sets. I can find RMSE and R squared (R^2, the coefficient of determination) from the output of my software (such as R's lm() function). ...
8
votes
1answer
3k views

Contribution of each covariate to a single prediction in a logistic regression model

Say, for example, that we have a logistic regression model which outputs the probability that a patient will develop a particular disease based on many covariates. We can get an idea of the ...
8
votes
1answer
6k views

How to manually calculate dfbetas

I am trying to replicate what the function dfbetas() does in R. dfbeta() is not an issue... Here is a set of vectors: ...
8
votes
2answers
4k views

Are t tests of coefficients in multiple regression post hoc tests?

In multiple regression, if a global F test is significant, then are t tests (or Wald tests) for the coefficients considered to be multiple comparisons and post hoc tests and should they be adjusted?
8
votes
2answers
773 views

What is the correct way to determine which features most contributed to the prediction of a given input vector?

I am using logistic regression for binary classification. I have a big data set (happens to be highly unbalanced: 19 : 1). So I use scikit-learn's ...
7
votes
4answers
347 views

Is the average of betas from Y ~ X and X ~ Y valid?

I am interested in the relationship between two time series variables: $Y$ and $X$. The two variables are related to each other, and it's not clear from theory which one causes the other. Given ...
7
votes
4answers
5k views

What is the difference between least square and pseudo-inverse techniques for Linear Regression?

I am wondering the difference between them. Basically they do the same job at the end finding coefficients of parameters, but they look just different the way we find the coefficients. To me, Least ...
7
votes
3answers
2k views

Explicit solution for linear regression with two predictors

I have some samples of data of the form $x,y$ and $z=f(x,y)$. I wish to fit a plane $$z = Ax + By + C$$ to the data with the smallest mean square errors. I have found an "answer" in section 3 of this ...
7
votes
1answer
2k views

Significance of individual coefficients vs Significance of both

This was a question I read from google quantitative analyst interview on glassdoor: If each of the two coefficient estimates in a regression model is statistically significant, do you expect the test ...
7
votes
3answers
6k views

Testing whether two regression coefficients are significantly different

I'm hoping somebody can help me out with this question. For a study I did a path analysis, which looks like this: IV --> Mediatior --> DV IV --> ...
7
votes
3answers
137 views

Converting the beta coefficient from matrix to scalar notation in OLS regression

I've found for my econometrics exams that if I forget the scalar notation, I can often save myself by remembering the matrix notation and working backwards. However, the following confused me. Given ...
7
votes
2answers
12k views

When to use Ridge regression and Lasso regression. What can be achieved while using these techniques rather than the linear regression model

I am looking forward to learning more about the regularized regression techniques like Ridge and Lasso regression. I would like to know what can be achieved by using these techniques when compared to ...
7
votes
1answer
3k views

How to interpret log-log regression coefficients for other than 1 or 10 percent change?

I have read many threads here on how to interpret coefficients in a regression where the predictor and the dependent variable are log-transformed. Most give an answer for a one or ten percent change. ...
7
votes
1answer
681 views

How is the determinant of $(X'X)$ related to variance?

I'm working on a problem (and actually have the answer) but I don't know why this is the answer, can someone explain this equality?. It has to do with the the determinant of the partitioned matrix $(X'...
7
votes
1answer
88 views

What is the difference between using random intercepts and slopes instead of separate regressions per subject?

I have recorded a DV and IV of 20 participants. The IV is a repeated measure, and my goal is to see how variation in the IV can explain variations in the DV. More specifically, I want a beta ...
7
votes
1answer
5k views

Interpretation of interaction term

I have a model: $$ \ln({\rm earnings}) = a+b_1{\rm female}+b_2{\rm white}+b_3{\rm female}\times{\rm white} $$ ${\rm female}$ and ${\rm white}$ are dummy variables. I have interpreted $b_1$ and $b_2$: ...
7
votes
2answers
10k views

How do I get coefficients of a random forest model?

I am using randomForest to generate a model, and at the end I don't know how I can get the final coefficients that the model is fitting. I know that for linear ...
7
votes
1answer
3k views

What are the (best) methods for multiple comparisons correction with bootstrap for multiple glm models?

See the related, but old question: Correcting p values for multiple tests where tests are correlated (genetics). Multiple comparison methods based on bootstrap have the advantage of taking account of ...
7
votes
2answers
3k views

Linear regression without intercept - sampling variance of coefficient

I am comparing linear regression with and without intercept for the general sampling case. For this, I have $n$ samples of two correlated random variables $X \sim N\left(0,\sigma_X^2\right)$ and $Y \...
7
votes
1answer
8k views

Sampling distribution of regression coefficients for normally distributed random variables

Based on $N$ realizations of two random variables $X \sim N(0,\sigma_X^2)$ and $Y \sim N(0, \sigma_Y^2)$ with correlation $\rho$, I conduct a simple linear regression $Y = \beta_0 + X\beta_1 + \...
7
votes
1answer
217 views

Interpreting regression coefficients based on Andrew Gelman's re-scaling method

I have two predictors in a binary logistic regression model: One binary and one continuous. My primary goal is to compare the coefficients of the two predictors within the same model. I have come ...
6
votes
4answers
900 views

Why is my regression insignificant when I merge data that produced two significant regressions?

Sorry for the confusing title, I think this is a general statistics question, but I'm working in R. I have a combined dataset of two samples from different countries (n=240 and n=1,010), and when I ...
6
votes
5answers
10k views

Coefficient changes sign when adding a variable in logistic regression

In my logistic regression the sign of coefficients of a variable (location distance of an amenity) changes based on other variables (with time -ve, with travel distance +ve) in the model. When the ...
6
votes
4answers
21k views

How to interpret standardized regression coefficients and p-values in multiple regression?

I've been using R to analyze my data (as shown in example below) and lm.beta from the QuantPsyc package to get the standardized ...
6
votes
3answers
2k views

How to interpret coefficients of $x$ and $x^2$ in same regression

If I have the below functional form for an OLS regression, how do I interpret the $x$ and $x^2$? I cannot interpret them separately, correct? Do I interpret them as a summation of the two coefficients,...
6
votes
2answers
47k views

Extract standard errors of coefficient linear regression R [duplicate]

Possible Duplicate: How do I reference a regression model's coefficient's standard errors? If I have a dataset: ...
6
votes
2answers
4k views

Linear regression with log transformed data - large error [duplicate]

I have a set of data which is has a very large positive skew, and has been transformed using a logarithm. I wish to predict one variable from another using the lm ...
6
votes
2answers
436 views

Invariance of results when scaling explanatory variables in logistic regression, is there a proof?

There is a standard result for linear regression that the regression coefficients are given by $$\mathbf{\beta}=(\mathbf{X^T X})^{-1}\mathbf{X^T y}$$ or $(\mathbf{X^T X})\mathbf{\beta}=\mathbf{X^T ...
6
votes
1answer
4k views

Can the coefficients of dummy variables be more than 1 or less than 0?

Can coefficients of dummy variables be more than $1$ or less than $0$? I am getting coefficients ranging from $-6$ to $-16$. I am specifically asking about the coefficients of dummy variables, not ...
6
votes
3answers
2k views

In linear regression, what does $\beta_1 = 0$ really mean?

If granted omniscience and we know that $\beta_1$ in a multiple linear regression model is truly 0, what does that mean in words (and math notation)? The model is: $Y = \beta_0 + \beta_1X_1 + \...
6
votes
2answers
670 views

What is the intuition behind getting a slope distribution in linear regression?

If I understand it correctly, linear regression finds one best fitting line for the given data. It can do it either by using calculus and solving for intercept and slope equations or it can solve it ...
6
votes
3answers
33k views

How to interpret a logistic regression model with all negative coefficient?

I have 4 predictors, and 1 binary response. I fitted a logistic regression model. A strange thing is that all the coefficient of the model are negative. Is that possible? Probably I did something ...
6
votes
1answer
5k views

How can I interpret coefficients of categorical predictors in the negative binomial regression model?

I used some categorical variables as predictors to a negative binomial model. The dependent variable is numerical. I used glm.nb in R and the results show relative coefficients of one category ...
6
votes
2answers
35k views

How to interpret Quadratic Terms

I'm answering a practice exam questions, and having trouble with one on quadratic terms. Could someone give me a quick summery of 1) why they are sometimes included? 2) How to interpret them? In ...
6
votes
2answers
6k views

What is the interpretation of scaled regression coefficients when only the predictors are scaled?

I'm running a model with 2 continuous predictors (x1, x2) and 1 continuous outcome variable (y). The results show that both of the slopes are significant, as well as the intercept, with no significant ...
6
votes
2answers
343 views

Positive Poisson regression: what is the effect on the model of shifting vs truncating?

Let's say that you were wanting to model how many times someone had to take a certain test before passing (depending on a range of predictors like practice, mock tests taken, classes attended, etc.). ...
6
votes
2answers
202 views

Inference and predictive models

Some people in my line of work are interested in the coefficients that results from developing predictive regression models(prm). I’m somewhat reluctant to use these coefficients to explain an ...
6
votes
1answer
800 views

why does R rlm {MASS} return different coefficients almost each time it is called?

I'm noticed that the rlm {MASS} returns almost every time different coefficients, even though I'm using the same parameters and the same data set I'm calling: <...
6
votes
1answer
756 views

R: regression coefficients and lubridate

I'm getting some odd coefficients when I apply lm to dates that have been processed and rounded using the lubridate package. ...
6
votes
1answer
2k views

Cohen's d from regression coefficient?

Is it appropriate to calculate Cohen's d (effect size) from the regression coefficient of an independent categorical variable? My coefficient represents participation in an intervention (treatment ...
6
votes
2answers
761 views

How do I examine biasedness in nonlinear regression?

I have the nonlinear regression model $$y_i=\beta_0+\beta_1 x_{1i}+e^{\beta_2 x_{2i}}+u_i,\quad i=1,2,\ldots,n,$$ and the least squares assumptions are satisfied (see below). Let $\beta=(\beta_0,\...
6
votes
1answer
8k views

How to get the standardized beta coefficients from glm.nb regression in R?

I'm working in R, using glm.nb (of the MASS package) to model count data with a negative binomial regression model. I'd like to get the standardized (beta) coefficients from the model, but am given ...
6
votes
1answer
136 views

regression coefficient on sum of regressors

Say I have the true linear model with normal errors: $y = \beta_1 X_1 + \beta_2 X_2 + \epsilon$ However, I only observe $Z = X_1 + X_2$, so I estimate instead: $y = \delta (X_1 + X_2) + e$, can I ...