Skip to main content
46 votes

Assumptions of multiple regression: how is normality assumption different from constant variance assumption?

1. Normal distribution of residuals: The normality condition comes into play when you're trying to get confidence intervals and/or p-values. $\varepsilon\vert X\sim N (0,\sigma^2 I_n)$ is not a ...
Antoni Parellada's user avatar
43 votes
Accepted

Why are survival times assumed to be exponentially distributed?

Exponential distributions are often used to model survival times because they are the simplest distributions that can be used to characterize survival / reliability data. This is because they are ...
klumbard's user avatar
  • 1,401
30 votes

Family of GLM represents the distribution of the response variable or residuals?

The family argument for glm models determines the distribution family for the conditional distribution of the response, not of the residuals (except for the quasi-models). Look at this way: For the ...
kjetil b halvorsen's user avatar
23 votes

Why are survival times assumed to be exponentially distributed?

To add a bit of mathematical intuition behind how exponents pop up in survival distributions: The probability density of a survival variable is $f(t) = h(t)S(t)$, where $h(t)$ is the current hazard ...
juod's user avatar
  • 2,240
23 votes

Family of GLM represents the distribution of the response variable or residuals?

Further to Kjetil's excellent answer, I wanted to add some specific examples to help clarify the meaning of a conditional distribution, which can be a bit of an elusive concept. Let's say you took a ...
Isabella Ghement's user avatar
22 votes
Accepted

Are 50% confidence intervals more robustly estimated than 95% confidence intervals?

This answer analyzes the meaning of the quotation and offers the results of a simulation study to illustrate it and help understand what it might be trying to say. The study can easily be extended by ...
whuber's user avatar
  • 328k
22 votes
Accepted

Multicollinearity and predictive performance

Let's assume that you have trained a model on a training dataset, and want to predict some values in a test/holdout dataset. Multicollinearity in your training dataset should only reduce predictive ...
mkt's user avatar
  • 18.9k
22 votes
Accepted

Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?

The operational meaning of the IID condition is given by the celebrated "representation theorem" of Bruno de Finetti (which, in my humble opinion, is one of the greatest innovations of ...
Ben's user avatar
  • 128k
20 votes

Why does including $x\ln(x)$ interaction term in logistic regression model helps to assess linearity assumption?

Box and Tidwell (1962) [1] presented a somewhat general approach for estimating transformations of the individual predictors (IVs), and work through the specific case of estimating power ...
Glen_b's user avatar
  • 286k
20 votes
Accepted

What is the need of assumptions in linear regression?

You are correct - you do not need to satisfy these assumptions to fit a least squares line to the points. You need these assumptions to interpret the results. For example, assuming there was no ...
rinspy's user avatar
  • 3,370
19 votes

What is a complete list of the usual assumptions for linear regression?

The following diagrams show which assumptions are required to get which implications in the finite and asymptotic scenarios. Linear Regression Assumptions: Key Points Generally the assumptions can ...
DVL's user avatar
  • 291
19 votes
Accepted

Why do I see a pattern in the residuals in this well specified model?

How close the residuals at specific $x$ values are to zero depends on the sample size. Now, the sample size in real examples will be whatever it is, so there's not much use saying it should be bigger, ...
Thomas Lumley's user avatar
19 votes

Which OLS assumptions are colliders violating?

I will assume models without intercepts to have shorter notation. Say the structural causal model is \begin{aligned} Y&=\beta_1X+u, \\ Z&=\gamma_1X+\gamma_2Y+v, \\ X&=w \end{aligned} with $...
Richard Hardy's user avatar
19 votes
Accepted

For linear regression, why do people usually standardize the X variables and log transform Y variables to make them normally distributed?

All 3 of your points are incorrect. The log transformation does not necessarily make the dependent variable normally distributed. But more importantly, it doesn't matter if the dependent variable is ...
mkt's user avatar
  • 18.9k
18 votes
Accepted

Is there i.i.d. assumption on logistic regression?

From your previous question you learned that GLM is described in terms of probability distribution, linear predictor $\eta$ and link function $g$ and is described as $$ \begin{align} \eta &= X\...
Tim's user avatar
  • 140k
18 votes

What is the need of assumptions in linear regression?

Try the image of Anscombe's quartet from Wikipedia to get an idea of some of the potential issues with interpreting linear regression when some of those assumptions are clearly false: most of the ...
Henry's user avatar
  • 40.5k
17 votes
Accepted

What are the consequences of having non-constant variance in the error terms in linear regression?

The consequences of heteroscedasticity are: The ordinary least squares (OLS) estimator $\hat{\mathbf{b}} = \left(X'X \right)X'\mathbf{y}$ is still consistent but it is no longer efficient. The ...
Matthew Gunn's user avatar
  • 22.6k
17 votes

Does regression work on data that isn't normally distributed?

A regression analysis assumes that the data is normally distributed conditioned on the variables in the regression model. That is, if this is the regression model: $$y=X\beta+\varepsilon$$ where $X$ ...
Ruben van Bergen's user avatar
15 votes

How should I check the assumption of linearity to the logit for the continuous independent variables in logistic regression analysis?

Logistic regression does NOT assume a linear relationship between the dependent and independent variables. It does assume a linear relationship between the log odds of the dependent variable and the ...
user114667's user avatar
15 votes

Are 50% confidence intervals more robustly estimated than 95% confidence intervals?

This is an interesting idea, and I can see how it is intuitively compelling, but I think it is too vague to be true or false. Here are a couple of questions I would want the commenter to clear up: ...
gung - Reinstate Monica's user avatar
15 votes

Is the inference from a parametric test valid when the population distribution is not normal?

I need to correct a number of mistaken or partly misplaced ideas in the question first (as well as some that aren't in the question but are commonly seen and may be indirectly influencing the way you ...
Glen_b's user avatar
  • 286k
15 votes
Accepted

What are the k-means algorithm assumptions?

This is a complicated question, as I believe that the role of model assumptions in statistics is generally widely misunderstood, and the situation for k-means is even less clear than for many other ...
Christian Hennig's user avatar
14 votes

What does "independent observations" mean?

In probability theory, statistical independence (which is not the same as causal independence) is defined as your property (3), but (1) follows as a consequence$\dagger$. The events $\mathcal{A}$ and ...
Ben's user avatar
  • 128k
14 votes

Why linear regression has assumption on residual but generalized linear model has assumptions on response?

The assumptions are not inconsistent. If, for $i = 1, \ldots, n$, you assume $$ Y_i = \beta_0 + \beta_1 X_{i1} + \ldots + \beta_k X_{ik} + \epsilon_i, $$ with the errors $\epsilon_i$ being normally ...
mark999's user avatar
  • 3,288
14 votes
Accepted

Why linear regression has assumption on residual but generalized linear model has assumptions on response?

Simple linear regression having Gaussian errors is a very nice attribute that does not generalize to generalized linear models. In generalized linear models, the response follows some given ...
Cliff AB's user avatar
  • 21.4k
14 votes

Instrumental variable exclusion restriction

There are two criteria for good instruments: The instrument $z$ is correlated with the endogenous variable $x$ (relevance). The instrument $z$ affects dependent variable $y$ only through $x$. In ...
ChrisP's user avatar
  • 268
14 votes
Accepted

What are the assumptions for applying a quantile regression model?

Quantile regression assumes the normal regression assumptions of linearity and additivity (unless you add more terms to the model) independence of observations very large sample size, as quantile ...
Frank Harrell's user avatar
14 votes

Model assumptions - not worth the effort?

Well, there is some truth to it, but it's problematic. It would lower the publication rate. OK. But getting rid of bad publications is a good thing. Too much garbage is published. Checking model ...
Peter Flom's user avatar
  • 125k
13 votes

Why are survival times assumed to be exponentially distributed?

You'll almost certainly want to look at reliability engineering and predictions for thorough analyses of survival times. Within that, there are a few distributions which get used often: The Weibull (...
fectin's user avatar
  • 271
13 votes
Accepted

Kruskal-Wallis test: assumption testing and interpretation of the results

The KW test (also the Mann-Whitney U-test) is essentially always a test for stochastic dominance. What that means is it is testing to see if there exists at least one group such that you would ...
gung - Reinstate Monica's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible