2
$\begingroup$

Background

Loosely adapting Angrist & Pischke Mostly Harmless Econometrics (A&P) section 3.1, suppose we have a conditional expectation function (CEF) $\mathbb{E}\left[Y_i|X_i\right]$ defined such that $$ \mathbb{E}\left[Y_i|X_i=x\right]\equiv\int t f_y\left(t|X_i=x\right)dt $$ A&P prove that given the OLS problem (3.1.2) $$ \beta \equiv argmin_b \mathbb{E}\left[\left(Y_i-X_i' b\right)^2\right] $$ $X_i'\beta$ provides the minimum least squared error to the CEF, that is (3.1.4) $$ \beta = argmin_b \mathbb{E}\left\{\left(\mathbb{E}\left[Y_i|X_i\right]-X_i'b\right)^2\right\} $$ I won't walk through their proof here since I'm basically going to follow it as part of the question below.

Question

Can the above be adapted to motivate the use of non-linear least squares?

Here is my attempt. Consider the below non-linear least-squares problem $$ \vartheta\equiv argmin_\theta \mathbb{E}\left[\left(Y_i-g(X_i,\theta)\right)^2\right] $$ where $g(X_i,θ)$ is assumed continuous but not necessarily differentiable.

Then adapting A&P 3.1.4 and its corresponding proof, it is also true that $g(X_i,\vartheta)$ provides the minimum least-squares approximation to $\mathbb{E}\left[Y_i|X_i\right]$ among all parameter choices, that is, $$ \vartheta = argmin_\theta \mathbb{E}\left[\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)^2\right] $$ This is true because $$ \begin{aligned} \left(Y_i-g(X_i,\theta)\right)^2 &= \left[\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)+\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)\right]^2 \\ &=\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)^2\\ &+\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)^2\\ &+\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right) \end{aligned} $$ The first term of the final expansion is not a function of the parameters $\theta$. The last term is zero by iterated expectations $$ \begin{aligned} &\mathbb{E}\left\{\left(Y_i-\mathbb{E}\left[Y_i|X_i\right]\right)\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)\right\}\\ &= \mathbb{E}\left\{\left(\mathbb{E}\left[Y_i|X_i\right]-\mathbb{E}\left[Y_i|X_i\right]\right)\left(\mathbb{E}\left[Y_i|X_i\right]-g(X_i,\theta)\right)\right\}\\ &=0 \end{aligned} $$

Is the above reasoning correct? Does it motivate non-linear least squares?

Also would be great to have a citation, but any help is appreciated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.