Suppose we obtain a linear regression model through least square minimization: $$ \hat y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 $$

Suppose $x_1 \sim N(0, \sigma_1^2)$, $x_2 \sim Ber(0.5)$.

Is it possible for me to obtain an expression / approximation for the mutual information between $\hat y$ and $x_2$? For example: $$ I(\hat Y, X_2) = H(\hat Y) - H(\hat Y|X_2) $$

Any hints / useful source link are appreciated! Thanks in advance!

  • $\begingroup$ And how will the problem change if $x_2 \sim N(0, \sigma_2^2)$? $\endgroup$ – Piggy Wenzhou Feb 23 at 5:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.