I have the nonlinear regression model $$y_i=\beta_0+\beta_1 x_{1i}+e^{\beta_2 x_{2i}}+u_i,\quad i=1,2,\ldots,n,$$ and the least squares assumptions are satisfied (see below). Let $\beta=(\beta_0,\beta_1,\beta_2)$.
Given this model, I want to examine the bias of the nonlinear least squares estimates of $\beta$. This estimate is given by the global solution to the minimization problem $$\min_{\beta_0,\beta_1,\beta_2}\sum_i(y_i-\beta_0-\beta_1 x_{1i}-e^{\beta_2 x_{2i}})^2.$$ The solution may be denoted by $\hat{\beta}=(\hat{\beta}_0,\hat{\beta}_1,\hat{\beta}_2)$.
(One may note that the first order conditions to the minimization problem are
\begin{cases} \sum_i(y_i-\beta_0-\beta_1 x_{1i}-e^{\beta_2 x_{2i}})&=0;\\ \sum_i(y_i-\beta_0-\beta_1 x_{1i}-e^{\beta_2 x_{2i}})x_{1i}&=0;\\ \sum_i(y_i-\beta_0-\beta_1 x_{1i}-e^{\beta_2 x_{2i}})e^{\beta_2 x_{2i}}&=0; \end{cases}
and add to this that I do not think we can find a analytic solution to this equation system. Thus, we may have to use computer algorithms to solve the system above.)
Now, I have two questions regarding $\hat{\beta}$.
A. Firstly, how do I examine the bias $E(\hat{\beta}-\beta)$? (Expectation is taken componentwise.)
B. Secondly, is the bias equal to zero for the intercept and the coefficient on $x_{1i}$? I.e., is it true that $$E(\hat{\beta}-\beta)=(0,0,E(\hat{\beta}_2-\beta_2))?$$ For the first question, A, I am looking for a method of investigation; it may involve using e.g. MATLAB or Mathematica. For the second question, B, I am looking for a proof or a counterexample. (Question B may be viewed as a subquestion to A.)
I am interested in this for it is often said that nonlinear regression introduce bias into the statistics. References for this statement are several, e.g., the Wikipedia article in English on nonlinear regression; page 51 in Nonlinear Regression, 2005, by G. A. F. Seber, C. J. Wild; Bias in Nonlinear Regression, Cook et al., Biometrika (1986), 73, 3, pp. 615-23; Bias in Nonlinear Estimation, M. J. Box, Journal of the Royal Statistical Society. Series B (Methodological) (1971), 33, 2, pp. 171-201; and the blog post Some Properties of Non-linear Least Squares by Dave Giles.
Least squares assumptions.
- For all $i$, the error term $u_i$ has conditional mean zero given $x_{1i}$ and $x_{2i}$, i.e. $E(u_i|x_{1i},x_{2i})=0$.
- $(y_i,x_{1i},x_{2i})$, $i=1,2,\ldots,n$, are i.i.d. draws from their joint distribution.
- Large outliers are unlikely, meaning that for each $i$, $y_i$, $x_{1i}$ and $x_{2i}$ have nonzero finite fourth moments.
- There is no perfect multicollinearity.