Recently I have learned about the practice of dichotomizing a continuous independent variable (or maybe even discretize it into more than 2 categories), and then run a predictive model (e.g. multiple linear regression, random forest). I am interested in its theoretical properties/justifications. I have tried to study the following simple situation, but couldn't work out the algebra.

Assume the true mode is $Y = a + 1_{\{X\ge c\}}b + \epsilon$ where $a, b,c, X$ are non-stochastic, and $\epsilon \sim \mathcal{N}\left(0,\sigma^2\right)$. Suppose we observe i.i.d. samples $\left(X_1,Y_1\right),\cdots,\left(X_n, Y_n\right)$. We mistakenly thought that the true model is $Y = a + Xb + \epsilon$ and calculated OLS estimator $\beta_{wrong}$. Let $\beta_{right}$ be the OLS estimator we would have calculated according to the correct model, and let $\beta = \begin{bmatrix}a & b \end{bmatrix}^T$. Is it true that $$\mathbb{E}\left[ \left\|\beta_{right} - \beta\right\|_2^2\right] \le \mathbb{E}\left[ \left\|\beta_{wrong} - \beta\right\|_2^2\right]\,?$$

I can find what LHS is. What would be the best way to approach the RHS? Should I look for an explicit expression of $\mathbb{E}\left[ \left\|\beta_{wrong} - \beta\right\|_2^2\right]$?


Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.