Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Reading from Logistic Regression slides by Erdem, before deriving the linear boundary equation from the GNB equation it says "What if we assume variance is independent of class i.e. $\sigma^2_{i,0}=\sigma^2_{i,1}$ and then put
$\prod\limits_{i=1}^{d}P(X_i|Y=0)P(Y=0)=\prod\limits_{i=1}^{d}P(X_i|Y=1)P(Y=1)$
what is that mean and what "variance is independent of class" assumption implies?
Let's say we have that variance is dependent of the class. We are talkin about the variance of the indipendent variable $X_i|Y=y_i$, which is a normal conditional PDF equal to 
You can clearly see that the variance of this variable depends on the class, infact we have the the $y$ index on the variance $\sigma_{i,y}$, meaning that variance could not be the same for all the classes (the classes anyway are two: $y = 0$, $y = 1$).
Practic example like the wikipedia one: We have a dependent variable which follows a Bernoulli rv where the students who pass the exam are the $1s$ the students who fail are the $0s$.
We have two normal random variables to explain the dependent variable: the $X_{1}|Y$ where the $x_{1,i}|y_{i}$ are the hours spent studying. The second random variable is $X_{2}|Y$ where the $x_{2,i}|y_{i}$ are the hours of sleeping. We run the model, we find the estimates and the decision boundary and:
we see that when a student fail the exam $y_i = 0$ there are no excuses: he failed because he studied very few hours, there's little randomness and we can assume that $\sigma^2_{i,y=0}=\epsilon$ where $\epsilon$ is a tiny variance.
we see that when a student pass the exam $y_i=1$ most of the cases he/she studied, but... there are many cases where he/she was lucky or has cheated, in fact many students who have low $x_{1,i}$ (they studied very few hours) have passed the exam. So in this case there is much randomness meaning that: $\sigma^2_{i,y=1}=\omega$, where $\omega > \epsilon$.
If we do not have the same variance the classification could not be perfect or efficient and we can't derive the linear boundary. If we assume that $\sigma^2_{i,y=0}=\sigma^2_{i,y=1}$ we can see that the conditional PDF loses the $y$ index, because in both cases of failing or passing the exam, we have the same variance.
So now with this assumption we can derive and obtain a decision boundary that is linear. If a distribution has a greater variance than the other one the classification could not be linear.
In your slides you see that you have a multivariate joint conditional PDF $X|Y=(X_{1},X_{2}|Y)$ with mean $M$ and variance $\Sigma$ that follows a normal distribution, where we have 2 information about the hours of study and the hours of sleeping. Like this one:
What I do not like in the following picture of your slides is that if we have a different variance of the classes, let's say $\Sigma_{1} < \Sigma_{2} $, we should see a wider circles in the image below on the right, that would mean that the distribution is "wider" or with great variance than when we have an $y_i=0$.
$\Sigma_1$ would be the variance when there is an $y_i=0$ and $\Sigma_2$ would be the variance when there is a $y_i=1$. Hope this helps
