73 votes
Accepted

How to interpret coefficients from a polynomial model fit?

My detailed answer is below, but the general (i.e. real) answer to this kind of question is: 1) experiment, mess around, look at the data, you can't break the computer no matter what you do, so ... ...
user avatar
  • 7,374
53 votes
Accepted

Is there a way to use the covariance matrix to find coefficients for multiple regression?

Yes, the covariance matrix of all the variables--explanatory and response--contains the information needed to find all the coefficients, provided an intercept (constant) term is included in the model. ...
user avatar
  • 287k
32 votes

How to compute the standard errors of a logistic regression's coefficients

The standard errors of the model coefficients are the square roots of the diagonal entries of the covariance matrix. Consider the following: Design matrix: $\textbf{X = }\begin{bmatrix} 1 & x_{...
user avatar
  • 321
31 votes
Accepted

interpreting estimates of cloglog logistic regression

With a complementary-log-log link function, it's not logistic regression -- the term "logistic" implies a logit link. It's still a binomial regression of course. the estimate of time is 0.015. Is ...
user avatar
  • 260k
29 votes
Accepted

Interpretation of betas when there are multiple categorical variables

You are right about the interpretation of the betas when there is a single categorical variable with $k$ levels. If there were multiple categorical variables (and there were no interaction term), the ...
user avatar
24 votes
Accepted

Why is my regression insignificant when I merge data that produced two significant regressions?

Without seeing your data, this is difficult to answer definitively. One possibility is that your datasets span different ranges of the independent variable. It is well-known that combining data ...
user avatar
21 votes
Accepted

Can standardized $\beta$ coefficients in linear regression be used to estimate the $R^2$?

The geometric interpretation of ordinary least squares regression provides the requisite insight. Most of what we need to know can be seen in the case of two regressors $x_1$ and $x_2$ with response $...
user avatar
  • 287k
21 votes

Negative relationship but regression analytics gives positive correlation coefficient

The correlation coefficient is $r$. $R^2$ is the square of $r$, and it is of course always positive, regardless of the sign of $r$. Taking the square root gives that $r= \pm 0.8489$, and since the ...
user avatar
19 votes
Accepted

Will larger correlation coefficient values result in greater slopes between x and y?

The answer is "not necessarily" — how correlated the variables are dictates how "noisy" the scatter plot is, but not how steep. In fact, the correlation and regression slope are ...
user avatar
  • 20.9k
19 votes

Can regression coefficients be higher than correlation coefficients?

For simple linear regression there is a relationship between slope and correlation: $\hat\beta_1 = r_{x,y}{s_y\over s_x}$ So the relationship of $\hat\beta_1$ and $r_{xy}$ is entirely dependent on ...
user avatar
  • 2,239
19 votes
Accepted

Frequentist perspective of regression coefficients and significance (coming from Bayesian background)?

A p value is the probability of observing a test statistic as or more extreme than the researcher's own test statistic, assuming the null hypothesis, and an assumed distribution model are both true. ...
user avatar
  • 26.4k
18 votes
Accepted

Linear regression with log transformed data - large error

If you say your model is ln(y) = b*ln(x) + a it is only part of your model. Your actual model includes an error term: $\ln y_i = b\cdot \ln x_i + a + \varepsilon_i$...
user avatar
  • 5,808
17 votes
Accepted

What's the difference between regression coefficients and partial regression coefficients?

"Partial regression coefficients" are the slope coefficients ($\beta_j$s) in a multiple regression model. By "regression coefficients" (i.e., without the "partial") the author means the slope ...
user avatar
17 votes
Accepted

Interpretation of LASSO regression coefficients

Are the LASSO coefficients interpreted in the same method as logistic regression? Let me rephrase: Are the LASSO coefficients interpreted in the same way as, for example, OLS maximum likelihood ...
user avatar
16 votes

Explicit solution for linear regression with two predictors

Elsewhere on this site, explicit solutions to the ordinary least squares regression $$\mathbb{E}(z_i) = A x_i + B y_i + C$$ are available in matrix form as $$(C,A,B)^\prime = (X^\prime X)^{-1} X^\...
user avatar
  • 287k
16 votes

Can the coefficients of dummy variables be more than 1 or less than 0?

Yes, coefficients of dummy variables can be more than one or less than zero. Remember that you can interpret that coefficient as the mean change in your response (dependent) variable when the dummy ...
user avatar
  • 20.9k
16 votes

When to use Ridge regression and Lasso regression. What can be achieved while using these techniques rather than the linear regression model

In short, ridge regression and lasso are regression techniques optimized for prediction, rather than inference. Normal regression gives you unbiased regression coefficients (maximum likelihood ...
user avatar
  • 1,128
16 votes

Why is my regression insignificant when I merge data that produced two significant regressions?

If your data looks something like this then the reason may be more obvious. Your two original regression lines would be almost parallel and look reasonably plausible but combined they produce a ...
user avatar
  • 31.2k
16 votes
Accepted

What is a random variable and what isn't in regression models

This post is an honest response to a common problem in the textbook presentation of regression, namely, the issue of what is random or fixed. Regression textbooks typically blithely state that the $X$ ...
user avatar
16 votes

Coefficient of 0.001 with p < 0.005

A simple thought experiment: suppose your predictor was a length, originally expressed in millimetres. If you express it instead in kilometres and fit the model again, you have not really changed ...
user avatar
  • 11.9k
15 votes

Do coefficients of logistic regression have a meaning?

The coefficients from the output do have a meaning, although it isn't very intuitive to most people and certainly not to me. That is why people change them to odds ratios. However, the log of the odds ...
user avatar
  • 94.5k
15 votes
Accepted

Compare the statistical significance of the difference between two polynomial regressions in R

...
user avatar
  • 5,808
15 votes
Accepted

Unable to get correct coefficients for logistic regression in simulated dataset

If you're trying to generate data from logistic regression's assumed data generating mechanism, your code does not do that. Logistic regression's data generating mechanism looks like $$ \eta = X\beta$$...
user avatar
14 votes

Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients

In short, I wouldn't use both the partial $R^2$ and the standardized coefficients in the same analysis, as they are not independent. I would argue that it is usually probably more intuitive to compare ...
user avatar
14 votes
Accepted

How to interpret Quadratic Terms

Lets consider an example (here I use Stata, but the logic works the same in any other package): ...
user avatar
  • 19.3k
14 votes

What is the difference between least square and pseudo-inverse techniques for Linear Regression?

In the context of linear regression, 'least squares' means that we want to find the coefficients that minimize the squared error. It doesn't specify how this minimization should be performed, and ...
user avatar
  • 29.7k
13 votes

Interpretation of log transformed predictor and/or response

The main purpose of linear regression is to estimate a mean difference of outcomes comparing adjacent levels of a regressor. There are many types of means. We are most familiar with the arithmetic ...
user avatar
  • 53k
13 votes
Accepted

How to compute the standard errors of a logistic regression's coefficients

Does your software give you a parameter covariance (or variance-covariance) matrix? If so, the standard errors are the square root of the diagonal of that matrix. You probably want to consult a ...
user avatar
  • 12.1k
13 votes

How to interpret coefficients of $x$ and $x^2$ in same regression

Such an equation describes a curved relationship between $y$ and $x$ - a parabola: (This particular set of parameters correspond to a minimum at $x= -\frac{_5}{^6}$, just off the left margin of this ...
user avatar
  • 260k

Only top scored, non community-wiki answers of a minimum length are eligible