Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression-coefficients]

The parameters of a regression model. Most commonly, the values by which the independent variables will be multiplied to get the predicted value of the dependent variable.

6
votes
1answer
508 views

What is the probability regression coefficient is larger than its OLS estimate

Consider a sample of 34 pairs of values $(x,y)$ for the regression equation $$ y_{i}=\alpha + \beta x_{i} + \epsilon_i . $$ Using linear regression (OLS), I got the estimate $\hat{\beta}=2.3$. What ...
6
votes
2answers
2k views

Feature Importance for Breast Cancer: Random Forests vs Logistic Regression

Assume I'm a doctor and I want to know which variables are most important to predict breast cancer (binary classification). Two different scientists each present me with a different feature importance ...
6
votes
1answer
756 views

Time varying coefficient in Cox model

I have a model for survival after an injury that is borderline passing the Schoenfeld test for the proportional hazards assumption (cox.zph() in R). However, ...
6
votes
2answers
1k views

Prior for the coefficients of a linear regression model

I have a linear regression model $\bf Y=\bf{X}\bf{\beta}+\epsilon$. I want to assign a prior on $\bf\beta$ in order to derive the posterior predictive model $p(y_{predictive}|\bf{y},\bf{X},\beta)$. ...
6
votes
1answer
397 views

Comparing coefficients of time series models

How do I test if two time series' coefficients differ significantly from one another? I feel like this should be pretty simple... should I just use the estimates/standard errors and calculate a Z-...
6
votes
1answer
6k views

How to interpret coefficients in a vector autoregressive model?

Can I interpret the coefficients in a VAR model in the same way as I do in a normal OLS regression?
6
votes
1answer
488 views

Standardized and unstandardized variables yield different results for mixed regression model

I have created two mixed regression models, one with raw unstandardized variables and the same model with standardized variables. When I convert the coefficients from the standardized variables I get ...
5
votes
1answer
8k views

Will larger correlation coefficient values result in greater slopes between x and y?

For example, if there are two data sets, and the first has a larger correlation between x and y than the second, does this mean the first data set has a greater slope than the second?
5
votes
2answers
4k views

Comparing coefficients in logistic regression

I have some problems I need help with. I am running a binary logistic regression. ...
5
votes
3answers
2k views

Should a predictor, significant on its own but not with other predictors, be included in an overall multinomial logistic regression?

I constructed a model via multinominal logistic regression analysis. The final model contains three predictors. All predictors are significant when they are the only predictors. However, the ...
5
votes
2answers
12k views

Relationship between regressing Y on X, and X on Y in logistic regression

Correlation and linear regression are sometimes distinguished in statistics books by saying that the former is symmetric and the latter is asymmetric in the following sense: in the case of correlation,...
5
votes
2answers
882 views

Distribution of linear combination of OLS regression coefficients

I have a simple linear OLS regression $Y_i = \alpha+ \beta_1 X_{1i} + \beta_2 X_{2i} + e_i$ where $e_i \sim N(0,\sigma)$. I have estimated the regression from the data and obtained estimates for my ...
5
votes
2answers
337 views

Population parameters of a regression

So this has really been bothering me and I was hoping for a (simple!) explanation if possible. Suppose I've specified a linear regression model: $$ Y = \beta_0 + \beta_1 X + \epsilon $$ And an ...
5
votes
2answers
3k views

How are partial regression slopes calculated in multiple regression?

I'm trying to understand how multiple regression statistically controls for the effects of other predictor variables when calculating partial regression slopes. In a multiple regression of Y~X1+X2, ...
5
votes
1answer
453 views

Why don't the results of testing $H_0 : \beta = 0$ and $H_0 : {\rm cor}(X,Y)=0$ agree?

I have 4 IVs in my model that directly effect the DV. The results of the correlation & regression analyses showed that: IV1&DV: Pearson Correlation Coefficient: insignificant Regression ...
5
votes
3answers
3k views

How to read the Interaction effect in multiple linear regression with continuous regressors?

If the interaction happens between a continuous and a discrete variable it is (if I'm not mistaken) relatively straightforward. ...
5
votes
4answers
3k views

Modeling prices with the Hedonic regression

I'm using the concept of Hedonic regression in order to model the prices for real estates. I'm having some trouble with my approach. What I have and what I do my data consists out of real estates ...
5
votes
2answers
5k views

Variance of slope

I have a bunch of data that I fit a linear regression to, and now I need to find the variance of my slope. Is there an analytical way to get this? If an example is necessary, consider this my data in ...
5
votes
1answer
2k views

Correctness of regression with ARIMA errors model and coefficient interpretation issues

I am trying to forecast electricity consumption in GWh for 2 years ahead (from June 2013 ahead), using R (the forecast package). For that purpose, I tried regression with ARIMA errors. I fitted the ...
5
votes
1answer
2k views

Zero regression coefficient when correlations are not zero

I don't really have a motivation for this - but I was thinking about this and couldn't work it out. Suppose I have a random variables $X$ and $Y$ which are correlated. Is it possible that the ...
5
votes
2answers
425 views

Regularized parameter overfitting the data (example)

Possible duplicate of (Why) do overfitted models tend to have large coefficients? How does regularization reduce overfitting? In the Coursera's machine learning course by Andrew Ng, I came across ...
5
votes
1answer
1k views

Relationship between least-squares regression and information theory

Is there a well-known relationship between least-squares regression and information theory? I've just started reading about information theory. It seems almost trivial to say that the regression ...
5
votes
1answer
13k views

Interpreting Principal Component Analysis output

If I have 50 variables in my PCA, I get a matrix of eigenvectors and eigenvalues out (I am using the MATLAB function eig). I have normalised the eigenvalues to sum ...
5
votes
1answer
812 views

Can the bias introduced by lasso change the sign of a coefficient?

L1 penalized regression introduces a bias on your regression model but decreases the variance. When this bias is introduced, is it possible that the coefficient of $B$ changes sign? This would ...
5
votes
1answer
3k views

Understanding over-dispersion as it relates to the Poisson and the Neg. Binomial

I am developing a Poisson-family glm model in R for a dataset that I have. This dataset has 650 entries with two measures of exposure. The model, though not that relevant to the question, is: $$\ln(E(...
5
votes
1answer
94 views

How can one produce many `p-values` in regression analysis?

In order to understand ANOVA and regression better, I read this: http://www.theanalysisfactor.com/why-anova-and-linear-regression-are-the-same-analysis/ It seems to make sense for the most part. The ...
5
votes
1answer
605 views

About the Randomized Dependence Coefficient

In the paper The Randomized Dependence Coefficient, authors introduce a novel dependence coefficient which seem to be quite generic and powerful compared to what is present in the literature. It is ...
5
votes
1answer
8k views

Difference between effect size (partial $R^2$) and coefficients [duplicate]

I am working with spoken language data and use linear models do determine the relationship between different phonological processes in my data. Background Measures of the regularity of syllable ...
5
votes
1answer
1k views

How would you report (in publication) the results of a linear model fit using the poly function in R?

@John recently pointed out to me that R's poly function produces less correlated values (more orthogonal) to fit polynomial predictors, i.e. the transformed ...
5
votes
1answer
12k views

Computing Confidence Intervals for Coefficients in Logistic Regression [duplicate]

After fitting a logistic regression model in R using model <- glm(y~x,family='binomial') I can obtain the confidence intervals for the fitted coefficients using <...
5
votes
1answer
985 views

High Pearson correlation, but very low coefficient in multiple regression analysis?

I have been running a few linear regression models to test the absolute and relative effect of several independent variables related to spending/investment on different tools on one measure of ...
5
votes
1answer
39 views

Ratio of Unbiased Estimators

If there is a linear regression model as follows: $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_3 + u$$ and we want to estimate the ratio of the slope coefficients: $$\theta = \frac{\beta_1}{\...
5
votes
1answer
410 views

Regression slope that increases persistently as my sample size increases

I found a peculiar feature in some data that I am analyzing and was wondering whether there was a technical term for this type of phenomenon and whether anyone has come across it before. I am doing a ...
5
votes
1answer
128 views

Need an Introduction to Generalized Non Linear Multiple Regression

I have been searching the internet for a generalized method for doing regression analysis on non linear data. My model can be represented as $$Y = \beta_0f(X_0) + \beta_1g(X_1) + ... + \beta_nz(X_n) +...
5
votes
1answer
445 views

Averaging LASSO coefficients for repeated random partitioning of data

Is it reasonable to average LASSO coefficients from repeated reshuffling of training/test sets? Suppose I randomly divide my data into testing & training sets, then within the training set use 10-...
5
votes
1answer
47 views

Pairwise comparisons of regression coefficients [duplicate]

I would like to know how to make quickly pairwise comparisons of regressions coefficients across three or more groups in R. Here is a small example: ...
5
votes
0answers
496 views

Can I calculate Cohen's $d$ from multiple regression coefficient?

Question: Is it appropriate to calculate Cohen's $d$ (effect size) from the regression coefficient of an independent categorical variable? Background: My regression coefficient represents ...
4
votes
4answers
15k views

Regression coefficients by group in R?

I have a dataframe with a group variable GRP (ranging from 1-100) and an X and Y for each one. I'd like to get a list of the regression intercepts and slopes for lm(Y~X) within each group. The ...
4
votes
2answers
2k views

Is the model wrong if a coefficient changes from minus in correlation table to plus in OLS?

Perhaps a very basic question but one that has me confused. Say, in a correlation table the relationship between A and the DV (B)...
4
votes
2answers
506 views

How to interpret logistic regression coefficient

How do I interpret a regression coefficient in a logistic regression with two predictors? $\hat{L} = -14.27+3.32(3)+0.88(7)$ My understanding is to take the anti-log of the coefficient, like $e^{3....
4
votes
2answers
113 views

Changing sign estimate manually

validated readers, Is someone allowed to manually change sign of an estimate (obtained through a OLS), if this is supported by an underlying theory? My first idea was that this is basically a ...
4
votes
2answers
13k views

difference between Nash-Sutcliffe efficiency and coefficient of determination

Both can be used for an assessment of model accuracy, but what is the difference? formula for coefficient of determination, or R²: with: SSres= sum (yi - fi)² and SStot = sum (yi - ymean)² y ...
4
votes
2answers
2k views

Density plot of parameter estimates from linear regression model

I am running a linear regression model in R: data(iris) fit1.iris = lm(Sepal.Length ~ Petal.Length+Petal.Width , data=iris) summary(fit1.iris) These are my ...
4
votes
3answers
5k views

Interpretation of coefficients in polynomial regression for predictive modeling

I am building a predictive model (binary target variable) in the financial services industry. One of the (many) potential predictors I am adding to the model is related to the customers checking ...
4
votes
2answers
2k views

Efficiency of beta estimates with heteroscedasticity

I need something clarified and that is when you have non-constant variance, estimates won't be biased but will be a problem when it comes to the S.E. formulas and efficiency. Therefore OLS estimates ...
4
votes
1answer
5k views

Using the normal equations to calculate coefficients in multiple linear regression

I am trying to understand how to get the coefficient of a multiple linear regression. The formula is: $b = (X'X)^{-1}(X')Y$ I try to calculate $b$ without package and with the ...
4
votes
1answer
10k views

How to interpret the significance code?

I'm studying for some time and I´m trying to do a logistic regression (using GLM in R) and now it´s extremely difficult to know what to do. I have a binary dependant variable and 15 independent ...
4
votes
1answer
224 views
4
votes
2answers
704 views

Interpretation of continuous variable in dummy-continuous interaction

Similar questions have been asked before, but all of them focus on the dummy or interaction term. Say run an OLS regression on the model: $\ln( housePrice )= \beta_1 \times pollutionLevel + \beta_2 ...
4
votes
1answer
12k views

Clarification: The covariance of intercept and slope in simple linear regression?

Help me understand this relatively simple (I think) concept: The covariance of the intercept ($\beta_0$) and the slope ($\beta_1$) in simple linear regression. Furthermore, what range of values ...