73
votes
Accepted
How to interpret coefficients from a polynomial model fit?
My detailed answer is below, but the general (i.e. real) answer to this kind of question is: 1) experiment, mess around, look at the data, you can't break the computer no matter what you do, so ... ...
- 7,374
53
votes
Accepted
Is there a way to use the covariance matrix to find coefficients for multiple regression?
Yes, the covariance matrix of all the variables--explanatory and response--contains the information needed to find all the coefficients, provided an intercept (constant) term is included in the model. ...
- 287k
32
votes
How to compute the standard errors of a logistic regression's coefficients
The standard errors of the model coefficients are the square roots of the diagonal entries of the covariance matrix. Consider the following:
Design matrix:
$\textbf{X = }\begin{bmatrix} 1 & x_{...
- 321
31
votes
Accepted
interpreting estimates of cloglog logistic regression
With a complementary-log-log link function, it's not logistic regression -- the term "logistic" implies a logit link. It's still a binomial regression of course.
the estimate of time is 0.015. Is ...
- 260k
29
votes
Accepted
Interpretation of betas when there are multiple categorical variables
You are right about the interpretation of the betas when there is a single categorical variable with $k$ levels. If there were multiple categorical variables (and there were no interaction term), the ...
24
votes
Accepted
Why is my regression insignificant when I merge data that produced two significant regressions?
Without seeing your data, this is difficult to answer definitively. One possibility is that your datasets span different ranges of the independent variable. It is well-known that combining data ...
- 1,768
21
votes
Accepted
Can standardized $\beta$ coefficients in linear regression be used to estimate the $R^2$?
The geometric interpretation of ordinary least squares regression provides the requisite insight.
Most of what we need to know can be seen in the case of two regressors $x_1$ and $x_2$ with response $...
- 287k
21
votes
Negative relationship but regression analytics gives positive correlation coefficient
The correlation coefficient is $r$. $R^2$ is the square of $r$, and it is of course always positive, regardless of the sign of $r$. Taking the square root gives that $r= \pm 0.8489$, and since the ...
- 3,888
19
votes
Accepted
Will larger correlation coefficient values result in greater slopes between x and y?
The answer is "not necessarily" — how correlated the variables are dictates how "noisy" the scatter plot is, but not how steep.
In fact, the correlation and regression slope are ...
- 20.9k
19
votes
Can regression coefficients be higher than correlation coefficients?
For simple linear regression there is a relationship between slope and correlation:
$\hat\beta_1 = r_{x,y}{s_y\over s_x}$
So the relationship of $\hat\beta_1$ and $r_{xy}$ is entirely dependent on ...
- 2,239
19
votes
Accepted
Frequentist perspective of regression coefficients and significance (coming from Bayesian background)?
A p value is the probability of observing a test statistic as or more extreme than the researcher's own test statistic, assuming the null hypothesis, and an assumed distribution model are both true.
...
- 26.4k
18
votes
Accepted
Linear regression with log transformed data - large error
If you say your model is ln(y) = b*ln(x) + a it is only part of your model. Your actual model includes an error term:
$\ln y_i = b\cdot \ln x_i + a + \varepsilon_i$...
- 5,808
17
votes
Accepted
What's the difference between regression coefficients and partial regression coefficients?
"Partial regression coefficients" are the slope coefficients ($\beta_j$s) in a multiple regression model. By "regression coefficients" (i.e., without the "partial") the author means the slope ...
17
votes
Accepted
Interpretation of LASSO regression coefficients
Are the LASSO coefficients interpreted in the same method as logistic regression?
Let me rephrase: Are the LASSO coefficients interpreted in the same way as, for example, OLS maximum likelihood ...
- 55.5k
16
votes
Explicit solution for linear regression with two predictors
Elsewhere on this site, explicit solutions to the ordinary least squares regression
$$\mathbb{E}(z_i) = A x_i + B y_i + C$$
are available in matrix form as
$$(C,A,B)^\prime = (X^\prime X)^{-1} X^\...
- 287k
16
votes
Can the coefficients of dummy variables be more than 1 or less than 0?
Yes, coefficients of dummy variables can be more than one or less than zero.
Remember that you can interpret that coefficient as the mean change in your response (dependent) variable when the dummy ...
- 20.9k
16
votes
When to use Ridge regression and Lasso regression. What can be achieved while using these techniques rather than the linear regression model
In short, ridge regression and lasso are regression techniques optimized for prediction, rather than inference.
Normal regression gives you unbiased regression coefficients (maximum likelihood ...
- 1,128
16
votes
Why is my regression insignificant when I merge data that produced two significant regressions?
If your data looks something like this then the reason may be more obvious. Your two original regression lines would be almost parallel and look reasonably plausible but combined they produce a ...
- 31.2k
16
votes
Accepted
What is a random variable and what isn't in regression models
This post is an honest response to a common problem in the textbook presentation of regression, namely, the issue of what is random or fixed. Regression textbooks typically blithely state that the $X$ ...
- 4,633
16
votes
Coefficient of 0.001 with p < 0.005
A simple thought experiment: suppose your predictor was a length, originally expressed in millimetres. If you express it instead in kilometres and fit the model again, you have not really changed ...
- 11.9k
15
votes
Do coefficients of logistic regression have a meaning?
The coefficients from the output do have a meaning, although it isn't very intuitive to most people and certainly not to me. That is why people change them to odds ratios. However, the log of the odds ...
- 94.5k
15
votes
Accepted
15
votes
Accepted
Unable to get correct coefficients for logistic regression in simulated dataset
If you're trying to generate data from logistic regression's assumed data generating mechanism, your code does not do that.
Logistic regression's data generating mechanism looks like
$$ \eta = X\beta$$...
- 25.8k
14
votes
Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients
In short, I wouldn't use both the partial $R^2$ and the standardized coefficients in the same analysis, as they are not independent. I would argue that it is usually probably more intuitive to compare ...
- 885
14
votes
Accepted
How to interpret Quadratic Terms
Lets consider an example (here I use Stata, but the logic works the same in any other package):
...
- 19.3k
14
votes
What is the difference between least square and pseudo-inverse techniques for Linear Regression?
In the context of linear regression, 'least squares' means that we want to find the coefficients that minimize the squared error. It doesn't specify how this minimization should be performed, and ...
- 29.7k
13
votes
Interpretation of log transformed predictor and/or response
The main purpose of linear regression is to estimate a mean difference of outcomes comparing adjacent levels of a regressor. There are many types of means. We are most familiar with the arithmetic ...
- 53k
13
votes
Accepted
How to compute the standard errors of a logistic regression's coefficients
Does your software give you a parameter covariance (or variance-covariance) matrix? If so, the standard errors are the square root of the diagonal of that matrix. You probably want to consult a ...
- 12.1k
13
votes
How to interpret coefficients of $x$ and $x^2$ in same regression
Such an equation describes a curved relationship between $y$ and $x$ - a parabola:
(This particular set of parameters correspond to a minimum at $x= -\frac{_5}{^6}$, just off the left margin of this ...
- 260k
Only top scored, non community-wiki answers of a minimum length are eligible