Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [regression-coefficients]

The parameters of a regression model. Most commonly, the values by which the independent variables will be multiplied to get the predicted value of the dependent variable.

Filter by
Sorted by
Tagged with
6
votes
1answer
8k views

How to get the standardized beta coefficients from glm.nb regression in R?

I'm working in R, using glm.nb (of the MASS package) to model count data with a negative binomial regression model. I'd like to get the standardized (beta) coefficients from the model, but am given ...
6
votes
1answer
161 views

regression coefficient on sum of regressors

Say I have the true linear model with normal errors: $y = \beta_1 X_1 + \beta_2 X_2 + \epsilon$ However, I only observe $Z = X_1 + X_2$, so I estimate instead: $y = \delta (X_1 + X_2) + e$, can I ...
6
votes
1answer
1k views

How to do permutation test on model coefficients when including an interaction term?

Given the following model as an example: $$Y=\beta_0+\beta_A\cdot A+\beta_B\cdot B+\beta_{AB}\cdot A \cdot B+\epsilon$$ In alternative notation: $$Y\sim A + B + A: B$$ The main question: When ...
6
votes
1answer
511 views

What is the probability regression coefficient is larger than its OLS estimate

Consider a sample of 34 pairs of values $(x,y)$ for the regression equation $$ y_{i}=\alpha + \beta x_{i} + \epsilon_i . $$ Using linear regression (OLS), I got the estimate $\hat{\beta}=2.3$. What ...
6
votes
1answer
93 views

Pairwise comparisons of regression coefficients [duplicate]

I would like to know how to make quickly pairwise comparisons of regressions coefficients across three or more groups in R. Here is a small example: ...
6
votes
2answers
2k views

Feature Importance for Breast Cancer: Random Forests vs Logistic Regression

Assume I'm a doctor and I want to know which variables are most important to predict breast cancer (binary classification). Two different scientists each present me with a different feature importance ...
6
votes
1answer
783 views

Time varying coefficient in Cox model

I have a model for survival after an injury that is borderline passing the Schoenfeld test for the proportional hazards assumption (cox.zph() in R). However, ...
6
votes
2answers
1k views

Prior for the coefficients of a linear regression model

I have a linear regression model $\bf Y=\bf{X}\bf{\beta}+\epsilon$. I want to assign a prior on $\bf\beta$ in order to derive the posterior predictive model $p(y_{predictive}|\bf{y},\bf{X},\beta)$. ...
6
votes
1answer
408 views

Comparing coefficients of time series models

How do I test if two time series' coefficients differ significantly from one another? I feel like this should be pretty simple... should I just use the estimates/standard errors and calculate a Z-...
6
votes
1answer
6k views

How to interpret coefficients in a vector autoregressive model?

Can I interpret the coefficients in a VAR model in the same way as I do in a normal OLS regression?
5
votes
1answer
9k views

Will larger correlation coefficient values result in greater slopes between x and y?

For example, if there are two data sets, and the first has a larger correlation between x and y than the second, does this mean the first data set has a greater slope than the second?
5
votes
2answers
4k views

Comparing coefficients in logistic regression

I have some problems I need help with. I am running a binary logistic regression. ...
5
votes
3answers
2k views

Should a predictor, significant on its own but not with other predictors, be included in an overall multinomial logistic regression?

I constructed a model via multinominal logistic regression analysis. The final model contains three predictors. All predictors are significant when they are the only predictors. However, the ...
5
votes
2answers
13k views

Relationship between regressing Y on X, and X on Y in logistic regression

Correlation and linear regression are sometimes distinguished in statistics books by saying that the former is symmetric and the latter is asymmetric in the following sense: in the case of correlation,...
5
votes
2answers
977 views

Distribution of linear combination of OLS regression coefficients

I have a simple linear OLS regression $Y_i = \alpha+ \beta_1 X_{1i} + \beta_2 X_{2i} + e_i$ where $e_i \sim N(0,\sigma)$. I have estimated the regression from the data and obtained estimates for my ...
5
votes
2answers
387 views

Population parameters of a regression

So this has really been bothering me and I was hoping for a (simple!) explanation if possible. Suppose I've specified a linear regression model: $$ Y = \beta_0 + \beta_1 X + \epsilon $$ And an ...
5
votes
3answers
5k views

Interpretation of coefficients in polynomial regression for predictive modeling

I am building a predictive model (binary target variable) in the financial services industry. One of the (many) potential predictors I am adding to the model is related to the customers checking ...
5
votes
2answers
6k views

Testing a regression coefficient against 1 rather than 0

Brief caveat- I haven't dusted off my stats knowledge since some university courses a few years ago, and I'm struggling with cobwebs. I have a model where a linear 1 to 1 relationship has been ...
5
votes
2answers
3k views

How are partial regression slopes calculated in multiple regression?

I'm trying to understand how multiple regression statistically controls for the effects of other predictor variables when calculating partial regression slopes. In a multiple regression of Y~X1+X2, ...
5
votes
1answer
457 views

Why don't the results of testing $H_0 : \beta = 0$ and $H_0 : {\rm cor}(X,Y)=0$ agree?

I have 4 IVs in my model that directly effect the DV. The results of the correlation & regression analyses showed that: IV1&DV: Pearson Correlation Coefficient: insignificant Regression ...
5
votes
3answers
3k views

How to read the Interaction effect in multiple linear regression with continuous regressors?

If the interaction happens between a continuous and a discrete variable it is (if I'm not mistaken) relatively straightforward. ...
5
votes
4answers
4k views

Modeling prices with the Hedonic regression

I'm using the concept of Hedonic regression in order to model the prices for real estates. I'm having some trouble with my approach. What I have and what I do my data consists out of real estates ...
5
votes
2answers
6k views

Variance of slope

I have a bunch of data that I fit a linear regression to, and now I need to find the variance of my slope. Is there an analytical way to get this? If an example is necessary, consider this my data in ...
5
votes
1answer
2k views

Correctness of regression with ARIMA errors model and coefficient interpretation issues

I am trying to forecast electricity consumption in GWh for 2 years ahead (from June 2013 ahead), using R (the forecast package). For that purpose, I tried regression with ARIMA errors. I fitted the ...
5
votes
1answer
2k views

Zero regression coefficient when correlations are not zero

I don't really have a motivation for this - but I was thinking about this and couldn't work it out. Suppose I have a random variables $X$ and $Y$ which are correlated. Is it possible that the ...
5
votes
4answers
208 views

Combining regression estimates by summing

I want to know if one can combine regression estimates from panel regressions when the new dependent variable is a sum of the dependent variables from previously estimated regressions. To be ...
5
votes
2answers
478 views

Regularized parameter overfitting the data (example)

Possible duplicate of (Why) do overfitted models tend to have large coefficients? How does regularization reduce overfitting? In the Coursera's machine learning course by Andrew Ng, I came across ...
5
votes
1answer
1k views

Relationship between least-squares regression and information theory

Is there a well-known relationship between least-squares regression and information theory? I've just started reading about information theory. It seems almost trivial to say that the regression ...
5
votes
1answer
13k views

Interpreting Principal Component Analysis output

If I have 50 variables in my PCA, I get a matrix of eigenvectors and eigenvalues out (I am using the MATLAB function eig). I have normalised the eigenvalues to sum ...
5
votes
1answer
869 views

Can the bias introduced by lasso change the sign of a coefficient?

L1 penalized regression introduces a bias on your regression model but decreases the variance. When this bias is introduced, is it possible that the coefficient of $B$ changes sign? This would ...
5
votes
1answer
3k views

Interpretation of coefficients in logistic regression output

I am doing logistic regression in R on a binary dependent variable with only one independent variable. I found the odd ratio as 0.99 for an outcomes. This can be shown in following. Odds ratio is ...
5
votes
1answer
3k views

Understanding over-dispersion as it relates to the Poisson and the Neg. Binomial

I am developing a Poisson-family glm model in R for a dataset that I have. This dataset has 650 entries with two measures of exposure. The model, though not that relevant to the question, is: $$\ln(E(...
5
votes
1answer
14k views

Computing Confidence Intervals for Coefficients in Logistic Regression [duplicate]

After fitting a logistic regression model in R using model <- glm(y~x,family='binomial') I can obtain the confidence intervals for the fitted coefficients using <...
5
votes
1answer
99 views

How can one produce many `p-values` in regression analysis?

In order to understand ANOVA and regression better, I read this: http://www.theanalysisfactor.com/why-anova-and-linear-regression-are-the-same-analysis/ It seems to make sense for the most part. The ...
5
votes
1answer
665 views

About the Randomized Dependence Coefficient

In the paper The Randomized Dependence Coefficient, authors introduce a novel dependence coefficient which seem to be quite generic and powerful compared to what is present in the literature. It is ...
5
votes
1answer
481 views

Why do we make a F-Test rather than a Beta-Test in ANOVAs?

When one performs an ANOVA, (s)he always end up calculating the observed F-ratio and comparing it to the appropriate F-distribution. From this post, I discovered that the coefficient of correlation $r^...
5
votes
1answer
8k views

Difference between effect size (partial $R^2$) and coefficients [duplicate]

I am working with spoken language data and use linear models do determine the relationship between different phonological processes in my data. Background Measures of the regularity of syllable ...
5
votes
1answer
1k views

How would you report (in publication) the results of a linear model fit using the poly function in R?

@John recently pointed out to me that R's poly function produces less correlated values (more orthogonal) to fit polynomial predictors, i.e. the transformed ...
5
votes
1answer
1k views

Interpreting a Quadratic Term in Binary Logistic Regression

Apologies in advance for my limited stats knowledge. I hope someone can help. I am trying to understand how to interpret the coefficients of both the linear and quadratic term in a binary logistic ...
5
votes
1answer
1k views

High Pearson correlation, but very low coefficient in multiple regression analysis?

I have been running a few linear regression models to test the absolute and relative effect of several independent variables related to spending/investment on different tools on one measure of ...
5
votes
1answer
4k views

How can you have significant correlations and insignificant coefficients?

I'm a psychology graduate, so I admit that statistics do not come naturally to me. However, I find them fascinating nonetheless. At the moment i'm struggling with regressions, or specifically in this ...
5
votes
1answer
50 views

Ratio of Unbiased Estimators

If there is a linear regression model as follows: $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_3 + u$$ and we want to estimate the ratio of the slope coefficients: $$\theta = \frac{\beta_1}{\...
5
votes
1answer
425 views

Regression slope that increases persistently as my sample size increases

I found a peculiar feature in some data that I am analyzing and was wondering whether there was a technical term for this type of phenomenon and whether anyone has come across it before. I am doing a ...
5
votes
1answer
129 views

Need an Introduction to Generalized Non Linear Multiple Regression

I have been searching the internet for a generalized method for doing regression analysis on non linear data. My model can be represented as $$Y = \beta_0f(X_0) + \beta_1g(X_1) + ... + \beta_nz(X_n) +...
5
votes
1answer
483 views

Averaging LASSO coefficients for repeated random partitioning of data

Is it reasonable to average LASSO coefficients from repeated reshuffling of training/test sets? Suppose I randomly divide my data into testing & training sets, then within the training set use 10-...
5
votes
0answers
257 views

Cross model comparison of quantile regression coefficients

I am looking for a way to compare coefficients obtained from quantile regression. The two surveyed models are nested, estimated on the same sample and for the same quantile. $$ Y = \beta_1X+\epsilon_2\...
5
votes
0answers
544 views

Can I calculate Cohen's $d$ from multiple regression coefficient?

Question: Is it appropriate to calculate Cohen's $d$ (effect size) from the regression coefficient of an independent categorical variable? Background: My regression coefficient represents ...
4
votes
4answers
16k views

Regression coefficients by group in R?

I have a dataframe with a group variable GRP (ranging from 1-100) and an X and Y for each one. I'd like to get a list of the regression intercepts and slopes for lm(Y~X) within each group. The ...
4
votes
5answers
285 views

How to interpret regression function with categorical variable?

I am trying to figure out how to interpret a regression function with no intercept and one categorical variable performed on a survey data. Each participant marks which actions, from a list of 25, ...
4
votes
2answers
4k views

In R, test whether coefficients in lm are different each to a given value (other than zero)

In R, is there a way to use the lm function to test for the hypothesis that the coefficients are different from a value other than zero? For instance, if the model ...