Recently I have learned about the practice of dichotomizing a continuous independent variable (or maybe even discretize it into more than 2 categories), and then run a predictive model (e.g. multiple linear regression, random forest). I am interested in its theoretical properties/justifications. I have tried to study the following simple situation, but couldn't work out the algebra.

Assume the true mode is $Y = a + 1_{\{X\ge c\}}b + \epsilon$ where $a, b,c, X$ are non-stochastic, and $\epsilon \sim \mathcal{N}\left(0,\sigma^2\right)$. Suppose we observe i.i.d. samples $\left(X_1,Y_1\right),\cdots,\left(X_n, Y_n\right)$. We mistakenly thought that the true model is $Y = a + Xb + \epsilon$ and calculated OLS estimator $\beta_{wrong}$. Let $\beta_{right}$ be the OLS estimator we would have calculated according to the correct model, and let $\beta = \begin{bmatrix}a & b \end{bmatrix}^T$. Is it true that $$\mathbb{E}\left[ \left\|\beta_{right} - \beta\right\|_2^2\right] \le \mathbb{E}\left[ \left\|\beta_{wrong} - \beta\right\|_2^2\right]\,?$$

I can find what LHS is. What would be the best way to approach the RHS? Should I look for an explicit expression of $\mathbb{E}\left[ \left\|\beta_{wrong} - \beta\right\|_2^2\right]$?


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.