Loosely adapting Angrist & Pischke Mostly Harmless Econometrics (A&P) section 3.1, suppose we have a conditional expectation function (CEF) $\mathbb{E}\left[Y_i|X_i\right]$ defined such that $$ \mathbb{E}\left[Y_i|X_i=x\right]\equiv\int t f_y\left(t|X_i=x\right)dt $$ A&P prove that given the OLS problem (3.1.2) $$ \beta \equiv argmin_b \mathbb{E}\left[\left(Y_i-X_i' b\right)^2\right] $$ $X_i'\beta$ provides the minimum least squared error to the CEF, that is (3.1.4) $$ \beta = argmin_b \mathbb{E}\left\{\left(\mathbb{E}\left[Y_i|X_i\right]-X_i'b\right)^2\right\} $$ I won't walk through their proof here since I'm basically going to follow it as part of the question below.


Can the above be adapted to motivate the use of non-linear least squares?

Here is my attempt. Consider the below non-linear least-squares problem $$ \vartheta\equiv argmin_\theta \mathbb{E}\left[\left(Y_i-g(X_i,\theta)\right)^2\right] $$ where $g(X_i,θ)$ is assumed continuous but not necessarily differentiable.

Then adapting A&P 3.1.4 and its corresponding proof, it is also true that $g(X_i,\vartheta)$ provides the minimum least-squares approximation to $\mathbb{E}\left[Y_i|X_i\right]$ among all parameter choices, that is, $$ \vartheta = argmin_\theta \mathbb{E}\left[\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)^2\right] $$ This is true because $$ \begin{aligned} \left(Y_i-g(X_i,\theta)\right)^2 &= \left[\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)+\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)\right]^2 \\ &=\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)^2\\ &+\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)^2\\ &+\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right) \end{aligned} $$ The first term of the final expansion is not a function of the parameters $\theta$. The last term is zero by iterated expectations $$ \begin{aligned} &\mathbb{E}\left\{\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)\right\}\\ &= \mathbb{E}\left\{\left(\mathbb{E}\left[Y_i|X_i\right]-\mathbb{E}\left[Y_i|X_i\right]\right)\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)\right\}\\ &=0 \end{aligned} $$

Is the above reasoning correct? Does it motivate non-linear least squares?

Also would be great to have a citation, but any help is appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.