Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite
1

With continuous data, a linear regression $Y=\beta_1+\beta_2X_2+u$ assumes that the error term is distributed N(0,$\sigma^2$)

1) Do we assume that Var(Y|x) is likewise ~N(0,$\sigma^2$)?

2) What is this error distribution in logistic regression? When the data is in the form of 1 record per case, where the "Y" is 1 or 0, is the error term distributed Bernoulli (i.e. variance is p(1-p) )) and when the data is in the form #successes out of #of trials, is it assumed binimial (i.e. variance is np(1-p)), where p is the probability that Y is 1?

share|improve this question
1  
You are not being precise.The model assumption is that the error terms are independent and identically distributed with a distribution that is N(0,σ$^2$) and is unrelated to the COVARIATE. What is Var(Y|x)? Are you conditioning on X$_2$ =x? Does the model assume the covariate is random in some way or so we assume that the covariate is fixed according to a design matrix? I think it is the latter and therefore Var(Y|X$_2$=x) is implied by the assumptions and does not need to be assumed. – Michael Chernick Sep 22 '12 at 3:28
@MichaelChernick Why does the model assume that $X_2$ is fixed? It certainly can be the case that it is fixed, but it can also be random. Nothing in the question implies either one to me. – Peter Flom Sep 22 '12 at 13:36
@PeterFlom I read into the question that linear regression with that assumed error distribution meant OLS which does require X$_2$ to be fixed and known. If someone has Deming regression (i.e. error in variables regression) it would be specified in the question. Looking at the answer Stat gave indicates that he interpeted the question that way too. – Michael Chernick Sep 22 '12 at 14:09
@Michael, I was assuming fixed X. – B_Miner Sep 22 '12 at 19:29
@B_Miner Thank you. That is what I expected. – Michael Chernick Sep 22 '12 at 19:36

1 Answer

up vote 7 down vote accepted

1) If $u$ has normal distribution i.e. $N(0,σ^2)$ then $Var(Y|X_2)=Var(β_1+β_2X_2)+Var(u)=0+σ^2=σ^2$, since $β_1+β_2X_2$ is not a random variable.

2) In the logistic regression, it is assumed that the errors follows a binomial distribution as mentioned here. It is better to write it as $Var(Y_j|X_j)=m_j.E[Y_j|X_j].(1-E[Y_j|X_j])=m_j\pi(X_j).(1-\pi(X_j))$, since those probabilities depend on $X_j$, as referenced here or in Applied Logistic Regression.

share|improve this answer
Stat, So, it is correct to say that the variance for the ith individual error, $e_i$, is $p_i$(1-$p_i$) which is equivalent to what you have shown assuming that there are more than 1 observation in the data with the same covariate pattern (i.e. else $m_j$ = 1 for all j)? – B_Miner Sep 22 '12 at 19:36
1  
Yes, this is correct. If $Y_i=p_i+e_i$ with $P(Y_i=1)=1-P(Y_i=0)=p_i$, then $e_i=1-p_i$ with probability $p_i$ or $e_i=-p_i$ with probability $1-p_i$. Hence $e_i$ has a distribution with mean $0$ and variance equal to $p_i(1-p_i)$. – Stat Sep 22 '12 at 20:29
One additional point here, Stat, we HAVE to assume that the X are fixed, non random for Var(Y|X) = Var(e) for both the cases of linear and logistic regression correct? – B_Miner Nov 20 '12 at 2:18

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.