Show Quadratic Discriminant Analysis (QDA) implies a logistic regression model

Question

I am attempting to show that QDA implies a logistic regression model of the form $\log(\frac{P(Y=1 \mid X = x)}{1-P(Y=1 \mid X = x)}) = \beta_0 + \beta_1x + \beta_2x^2$, where there is only one covariate x

Let $P(Y=1 \mid X = x) = p_1(x) = \frac{\cfrac{\pi_1}{\sqrt{2\pi\sigma_1}}exp(\cfrac{-(x-\mu_1)^2}{2\sigma_1^2})}{\sum_{l=1}^{K} \pi_l \cfrac{1}{\sqrt{2\pi\sigma_1}}exp(\cfrac{-(x-\mu_l)^2}{2\sigma_1^2})}$

Then, $\frac{p_1(x)}{1-p_1(x)} = \frac{\cfrac{\pi_1}{\sqrt{2\pi\sigma_1}}exp(\cfrac{-(x-\mu_1)^2}{2\sigma_1^2})}{ \sum_{l=1}^{K} \pi_l \cfrac{1}{\sqrt{2\pi\sigma_1}}exp(\cfrac{-(x-\mu_l)^2}{2\sigma_1^2}) -\cfrac{\pi_1}{\sqrt{2\pi\sigma_1}}exp(\cfrac{-(x-\mu_1)^2}{2\sigma_1^2})}$

Taking the log of the equation above, I can deduce the numerator to be: $\log(\pi_1)-\log(\sqrt{2\pi\sigma_1}) -\frac{(x-\mu_1)^2}{2\sigma_1^2}$

But, I do not know how to continue further. I am unsure as to how to take the log of the denominator, and make the terms simplify, so that I can yield the aforementioned quadratic form.

Answer 1

For your setting of a general QDA with only one covariate and two classes, following your notation we can first use Bayes theorem to denote:$$P(Y=1 \mid X = x)=p_1(x)=\frac{\pi_1f_1(x)}{\pi_2f_2(x)+\pi_1f_1(x)}$$, where $\pi_1,\pi_2$ are class priors and $f_1,f_2$ are class conditional density for class $1$ and $2$, respectively.

Now with this simple form to calculate the odds: $$\frac{p_1(x)}{1-p_1(x)} = \frac{\pi_1f_1(x)}{\pi_2f_2(x)}$$

Further applying the logit quantile function: $$\log\frac{p_1(x)}{1-p_1(x)} = \log\frac{\pi_1}{\pi_2}+\log\frac{f_1(x)}{f_2(x)}$$

Since the class conditional densities of QDA are assumed Gaussian with mean $\mu_1,\mu_2$ and variance $\sigma_1^2,\sigma_2^2$ for class $1$ and $2$, respectively, i.e., $f_1(x)=\frac{1}{\sqrt{2\pi\sigma_1^2}}\exp(\frac{-(x-\mu_1)^2}{2\sigma_1^2}), f_2(x)=\frac{1}{\sqrt{2\pi\sigma_2^2}}\exp(\frac{-(x-\mu_2)^2}{2\sigma_2^2})$, we can substitute and expand the RHS of above logit function as: $$\log\frac{\pi_1}{\pi_2}-\frac{1}{2}\log\frac{\sigma_1^2}{\sigma_2^2}-\frac{(x-\mu_1)^2}{2\sigma_1^2}+\frac{(x-\mu_2)^2}{2\sigma_2^2}$$

Expand the quadratic terms and combine like terms, the RHS of logit becomes: $$(\frac{1}{2\sigma_2^2}-\frac{1}{2\sigma_1^2})x^2+(\frac{\mu_2}{\sigma_2^2}-\frac{\mu_1}{\sigma_1^2})x+(\log\frac{\pi_1}{\pi_2}-\frac{1}{2}\log\frac{\sigma_1^2}{\sigma_2^2}+\frac{\mu_2^2}{2\sigma_2^2}-\frac{\mu_1^2}{2\sigma_1^2})$$

Now by comparing with your logistic regression model with an additional quadratic form in $x$, i.e., $\beta_0 + \beta_1x + \beta_2x^2$, we can easily match each parameter $\beta_i$ with the above derived RHS result. The trick is to delay your Gaussian densities substitution later to reduce the computation steps and complexity, and also it seems you made some typos about the Gaussian density function on the numerator position of its coefficient in your question.

Finally and perhaps importantly you can see clearly from above derivation that when the variances of the two class conditional densities are tied QDA simplifies to LDA (linear determinant analysis) with linear decision boundaries, which incidentally verifies the above derivation.