Questions tagged [regression-coefficients]

The parameters of a regression model. Most commonly, the values by which the independent variables will be multiplied to get the predicted value of the dependent variable.

4
votes
1answer
12k views

Clarification: The covariance of intercept and slope in simple linear regression?

Help me understand this relatively simple (I think) concept: The covariance of the intercept ($\beta_0$) and the slope ($\beta_1$) in simple linear regression. Furthermore, what range of values ...
4
votes
2answers
889 views

In logistic regression, does the lack of significance of the parameter estimates in a test sample indicate overfitting?

I am trying to build a logistic regression model where I have a dependent variable $y$ and independent variables $x_1$, $x_2$... $x_n$. $y$ can take only two values - 0 or 1. My original modelling ...
4
votes
1answer
121 views

Why do descriptive statistics contradict with regression coefficents?

I am estimating a binary logistic regression with L1 norm. According to the regression coefficients, the sign of x1's ...
4
votes
1answer
42 views

Construct a $95\%$ confidence interval for $5\beta_4$

Construct a $95\%$ confidence interval for $5\beta_4$. If this question were about $\beta_4$ without the $5$, I would absolutely know what to do. But I have to idea how the $5$ comes into play. I can'...
4
votes
1answer
440 views

How to interpret regression coefficient if predictor itself is on a negative scale?

I'm looking at effects of tree mortality (using "Biomass loss") on forest growth patterns. I incorporate loss into a mixed effects model like so (using lmer in R): ...
4
votes
1answer
7k views

Formula for standardized Regression Coefficients(derivation and intuition)

Here is the formula of standardized regression coefficients. I have two questions: 1)How do we derive this formula? 2)How can we understand intuitively this formula(I cannot understand why do we ...
4
votes
1answer
3k views

Interpretation of coefficients in logistic regression output

I am doing logistic regression in R on a binary dependent variable with only one independent variable. I found the odd ratio as 0.99 for an outcomes. This can be shown in following. Odds ratio is ...
4
votes
1answer
401 views

Sampling variance of regression intercept when there is no regressor

Suppose we have a model $y=\beta_0+u$, where $E(u)=0$ and $Var(u)=\sigma^2$. I get the unbiased estimator $\hat\beta_0$ is just $\bar y$. But how can I get the variance of $\hat\beta_0$? Is it correct ...
4
votes
1answer
473 views

Is there something called “mean coding” (like dummy coding & effect coding) in regression models?

When we perform a regression analysis with categorical predictors, we can use (1, 0), called "dummy coding". The coefficients in this case represent the deviation of the groups' means from the mean of ...
4
votes
1answer
168 views

How to interpret two observations that are otherwise identical in a regression model

I am confused trying to interpret how two observations are otherwise identical but differ by a dummy variable. For example if we have the following model with a factor variable race being White race ...
4
votes
1answer
2k views

Interpreting coefficient values in lasso regression

I am running a lasso regression function. I have about 45 features and I am predicting 1 dependent variable. After running lasso regression I get the coefficient values of the features. If I look at ...
4
votes
1answer
6k views

Testing a regression coefficient against 1 rather than 0

Brief caveat- I haven't dusted off my stats knowledge since some university courses a few years ago, and I'm struggling with cobwebs. I have a model where a linear 1 to 1 relationship has been ...
4
votes
1answer
1k views

Wald test and Likelihood ratio test, where do the confidence intervals on the regression coefficients come from?

So I'm trying to build my own Wald test and likelihood ratio test code within a machine learning pipeline. I can get the final fitted logistic regression coefficients from liblinear. I'm coding in ...
4
votes
2answers
1k views

Robust regression - a better understanding

I looked at robust regression for the first time today and I am a bit confused, comparing it to something like ordinary least squares and I am not sure if I am on the right track. I read a few ...
4
votes
1answer
2k views

Assessing fit with identity line in Q-Q plot

I'm using a QQ plot to asses the similarity between two observed distributions. I need a number that quantifies how much the quantile-quantile plot separates itself (is there a better word for this ...
4
votes
1answer
7k views

Not standardizing outcome, standardizing predictors only

I do understand the advantages of standardizing regression predictors to get standardized coefficients, in order to interpret the coefficients better. However, as I was reading multiple pages online, ...
4
votes
1answer
78 views

How can I test the significance of a non-linear function of regression coefficients in R?

Let's say I have an equation $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + ... + \beta_k X_k $, where $\beta_i$ represents an estimated coefficient and $X_i$ represent independent variables. How can I ...
4
votes
1answer
2k views

Understanding the formula of dfbetas

I'm referring to the formula used in the answer here. The numerator in the formula for dfbetas is straight forward: the difference between the value of the coefficient for a regression model that ...
4
votes
1answer
442 views

Why do we make a F-Test rather than a Beta-Test in ANOVAs?

When one performs an ANOVA, (s)he always end up calculating the observed F-ratio and comparing it to the appropriate F-distribution. From this post, I discovered that the coefficient of correlation $r^...
4
votes
1answer
236 views

Is the significance of difference in slopes equivalent to the significance of the slope of the difference of two series?

Say you have an independent variable, $x$, and two dependent variables $y_1$ and $y_2$. I want to calculate whether these two variables have a significantly different slope. I can do it by calculating ...
4
votes
1answer
787 views

How can I optimize coefficients of an arbitrary model?

This might be terribly easy but I'm probably lacking the keywords to search for. Assume the following (dummy) data: ...
4
votes
1answer
238 views

Combine 2 groups based on the regression coefficient

I have two groups of people, N=211 in group 1, N= 310 in group 2. I would like to combine these groups because they share a certain characteristic that I am interested in. The groups are used in a ...
4
votes
1answer
4k views

How can you have significant correlations and insignificant coefficients?

I'm a psychology graduate, so I admit that statistics do not come naturally to me. However, I find them fascinating nonetheless. At the moment i'm struggling with regressions, or specifically in this ...
4
votes
1answer
198 views

Uniqueness of $x'\beta$ even when $\mathbb{E}(x^Tx)$ is not invertible

As discussed in user25658's answer to this question, when one wants to compute $$ \beta = \mathbb{E}(x^Tx)^{-1} \mathbb{E}(x^TY) $$ but $\mathbb{E}(x^Tx)$ is not invertible, $\beta$ is not uniquely ...
4
votes
3answers
2k views

Efficiency and the number of regressors?

An econometrician told me that I shouldn't keep adding new variables to the model even if I have reason to believe they're relevant to the response variable, as it "reduces the efficiency of the other ...
4
votes
1answer
778 views

Interpreting a Quadratic Term in Binary Logistic Regression

Apologies in advance for my limited stats knowledge. I hope someone can help. I am trying to understand how to interpret the coefficients of both the linear and quadratic term in a binary logistic ...
4
votes
1answer
261 views

How to deal with missing coefficients while bootstrapping regressions

I'm using R boot() function to perform regression bootstrapping. When boot() resamples my data, can happen that some coefficients are missing, especially in the case of factor variables with many ...
4
votes
1answer
9k views

Interpreting intercept for the log model in linear regression in R for small predictor

I have a dataset. Assume that y is the dependent variable and x is the independent variable. My goals for this analysis is mainly on the following hypothesis: Expecting x=0 to imply y=0 Expecting ...
4
votes
2answers
3k views

Standard deviation of the sum of regression coefficients

I'm doing OLS estimation with an independent variable lagged as t-1, t-2, t-3, and t-4 (four beta coefficients). I would like to have the sum of these coefficients for interpreting the net impact of ...
4
votes
1answer
182 views

linear regression, symmetry of model does not lead to symmetry of coefficients

Experiment: You are given a large population of real numbers. For simplicity take the whole numbers from -n to n. Take two independent random samples x and y of size k and sort them (each one ...
4
votes
1answer
262 views

Poisson Regression and Negative Binomial regression results interpretation

I'm using Poisson Regression and Negative Binomial regression to estimate temporal trends. My understanding is that the coefficients are in log scale and they have to be translated to data-unit (count ...
4
votes
1answer
3k views

What is a “high” standard error (in logistic regression)?

I can't find in any statistics book what would start to be considered a large standard error of a regression coefficient. In my research, I have a group of a categorical variable with a small number ...
4
votes
1answer
3k views

Linear SVM feature weights interpretation. Binary classification, only positive feature values

I'm using clf = svm.SVC(kernel='linear') on a data set with only two classes $y \in \{-1, +1\}$ and the feature values of all samples are normalized between 0 and 1....
4
votes
1answer
331 views

Multivariate Bayesian Testing with an F-test

In Bayesian statistics a standard way to perform a Lindley significance test for the hypothesis $\theta=\theta_0$, where $\theta_0$ is the suggested value for $\theta$ at the $\alpha$ level of ...
4
votes
1answer
495 views

Adjusting regression coefficient for predictor error

I saw a famous review paper about intelligence, and the authors introduced a way to adjust the regression coefficient for predictor error. As many of you might know, if the predictor has a ...
4
votes
1answer
68 views

regression - does R2 only apply to measure linear regression performance?

Background According to Wiki: https://en.wikipedia.org/wiki/Coefficient_of_determination, $R^2$ is coefficient of determinant. The definition is $$ R^2 = 1 - \dfrac{SSE}{SST} $$ Since $SSE$ is ...
4
votes
0answers
234 views

Are these negative binomial regression results reasonable?

I did a negative binomial regression on a data set with 4 covariables. The count outcome has values up to 600. I did a mixture model with 2 components, also called a latent class model. However, I am ...
4
votes
0answers
186 views

R bayesglm: Estimates depends on order of variable

I did a logistic regression with bayesglm from package arm. I got different results depending on the order of the variables in the model: ...
4
votes
0answers
243 views

Cross model comparison of quantile regression coefficients

I am looking for a way to compare coefficients obtained from quantile regression. The two surveyed models are nested, estimated on the same sample and for the same quantile. $$ Y = \beta_1X+\epsilon_2\...
4
votes
0answers
48 views

Confidence interval for the increase in P(Y=1) from moving between 2 levels of a factor in logistic regression

I have a logistic regression model fit with one categorical variable $x$ that takes value in $\{1,2,3,4,5\}$. In R I have obtained the estimate and standard error for $\beta_0$ and $\beta_1$. The ...
4
votes
0answers
58 views

Efficient ways to partition rows of augmented design matrix $[X|y]$ into subsets with similar regression results?

Imagine I have $n$ observations on a regression model; are there any reasonably efficient methods for partitioning that into two (or more) roughly equally sized groups which almost reproduce the ...
4
votes
0answers
793 views

SARIMA, coefficients check

I would appreciate if someone could check the mathematical equation for the seasonal ARIMA (4,1,4) x (1,1,1) period 12 that I wrote. I have done it this way, but I am not really sure if it correct is. ...
4
votes
0answers
566 views

Aggregating Standard Errors for Predicted Probability Estimates

I obtain predicted values from a logistic regression for a certain outcome (e.g., mortality) at the hospital level – the data is at the patient level – and need to compute the average across hospitals....
4
votes
0answers
208 views

Backing out the standard deviation from information on baseline mean/s.d., and coefficient mean/s.d

I am trying to run a power calculation for a randomized control trial. For this I need a mean and standard deviation for our 'baseline'. There are papers out there which would have a mean and standard ...
3
votes
2answers
3k views

In R, test whether coefficients in lm are different each to a given value (other than zero)

In R, is there a way to use the lm function to test for the hypothesis that the coefficients are different from a value other than zero? For instance, if the model ...
3
votes
4answers
2k views

How to interpret regression equations with logarithms, based on log difference being approximate to percentage change?

$y = 4 + 2.5\,x + u$ For an increase of 1 unit of $X$ (that is, $X$ to $X+1$), we expect an increase $2.5$ units of $Y$ (that is, $Y$ to $Y+2.5$). Is that right? What if there's a/an $\ln$? $\ln(y)...
3
votes
1answer
296 views

Regression when response variable is a function

I have a set of data $(X_i,Y_i)$, $i=1,\ldots,n$ where $X$ and $Y$ are supposed to satisfy the following equation $$ y = \beta_0(1+x^2)^{\beta_1},\quad x>0, \quad\quad (1) $$ I am interested in ...
3
votes
2answers
58 views

Analyzing the results from a logistic regression

I'm new to logistic regression. Can you help me understand how to read this? Here's what I understand - For every +1 of continous_variable, the probability of ...
3
votes
2answers
377 views

Can I make a better linear model than this?

I am getting the below plot for my data, and relevant summary is as below:- ...
3
votes
1answer
172 views

Changing polynomial degree leads to changing p-values in OLS regression

I have a question about interpreting coefficient $p$-values when fitting a polynomial function with ordinary least squares. When I sequentially fit a linear, then quadratic, then cubic etc. ...