# Iterated estimation of Taylor series

Say your data generating process is given by the function $$y=f(x|\theta)$$, where $$y$$ and $$x$$ represent variables (data) and $$\theta$$ represent parameter(s). For convergence reasons (e.g. $$f(\cdot)$$ is highly non-linear on parameters and a GMM estimator does not converge), you decide to estimate a Taylor series expansion of $$f(\cdot)$$ around $$\theta=\theta_0$$. Let's denote this approximated function as $$y \approx g(x|\theta)_{\theta_0}$$.

Say you estimate $$\theta$$ in $$g(\cdot)$$ based on a random sample of $$\{y,x\}$$, and you get $$\hat\theta_1$$. Then, you recompute the Taylor series approximation around this point estimate (keeping the Taylor series order constant), and produce $$y \approx g(x|\theta)_{\hat\theta_1}$$. Then, you estimate again, yielding $$\hat\theta_2$$. You iterate until

$$(\hat\theta_n - \hat\theta_{n+1})^2 < \epsilon$$

for an arbitrary threshold $$\epsilon > 0$$.

Convergence (in terms of the optimisation criterion above) is of course of paramount importance. Notice that for an arbitrarily large $$\epsilon$$ there is always a solution, as long as $$\hat\theta$$ can be computed, which itself depends on the properties of $$g(\cdot)$$, e.g. on the order of the Taylor expansion; a linear model is always estimable, beyond trivial issues like multicolinearity.

My question is, is the method above a thing? I've searched for "iterated estimation of taylor series" on Google, in this forum and in Math.SE and cannot find anything about this. Maybe the method is just plainly wrong, e.g. convergence is not assured by any known theorem.

More details on the method

For instance, consider a CES production function:

$$Y = \left(\alpha K^\theta+ (1-\alpha)L^\theta\right)^{1/\theta}$$

where Y, L and K are variables, and $$\alpha$$ and $$\theta$$ are parameters. Assume we are particularly interested in estimating $$\theta$$.

So, you produce a first order Taylor series expansion of the log of $$Y$$, around $$\theta= \theta_0$$. The new formula (which is equivalent to the so-called translog production function when $$\theta_0 = 0$$) is:

$$ln(Y) \approx \frac{1}{\theta_0} ln\left(\alpha K^\theta_0+ (1-\alpha)L^\theta_0\right) + (\theta - \theta_0)\left[\frac{1}{\theta^2} ln\left(\alpha K^\theta_0+ (1-\alpha)L^\theta_0\right) + \frac{1}{\theta_0}\frac{\left(\alpha K^\theta_0 ln(K)+ (1-\alpha)L^\theta_0 ln(L)\right)}{\alpha K^\theta_0+ (1-\alpha)L^\theta_0} \right]$$

So, you estimate the above equation with a random sample of $$\{Y,L,K\}$$, using e.g. non-linear least squares, for a given arbitrary $$\theta_0$$. Importantly, $$\theta_0 \neq 0$$, because otherwise the equation above changes completely (see translog function in link). From this estimation, you obtain an estimate of $$\theta$$, $$\hat\theta_1$$. Then, you re-estimate the model assuming $$\theta_0 = \hat\theta_1$$ (so, a new Taylor series around a different value). Then, estimate the new equation, obtaining $$\hat\theta_2$$. Iterate until some convergence criterion is fulfilled.

• I am having difficulty understanding your procedure. Could you elaborate on what you mean by "estimate a Taylor series expansion of $f(\cdot)$ around $\theta=\theta_0$"? This notation seems explicitly to refer to an expansion in both arguments $x$ and $\theta,$ but by referencing only $\theta$ it appears to be a Taylor series only in the second argument, but AFAIK the only thing you can "estimate" is the behavior of $f$ in its first argument! – whuber Dec 7 '18 at 16:17
• @whuber I added an example. I hope it's clearer. – luchonacho Dec 7 '18 at 16:50
• I'm still confused, as I will explain. Please note that this is a first order series in $\theta.$ As such, I cannot see why it would require "non-linear" least squares, because ordinary least squares regression of $\log(Y)$ against $\alpha(1-\alpha)(\log(K/L))^2$ will fit it (assuming you know $\alpha$, which--although you refer to it as a "parameter"--seems to have been forgotten). And if you do use a higher order expansion in all parameters, it seems like your problem isn't any simpler than just performing a nonlinear fit to the desired model in the first place. – whuber Dec 7 '18 at 17:35
• That's exactly how many optimizers work. – whuber Dec 7 '18 at 20:21
• Try "Newton's Method" (en.wikipedia.org/wiki/Newton%27s_method_in_optimization) – jbowman Dec 9 '18 at 18:34

If you take the $$n^{\text{th}}$$ order Taylor expansion of $$\theta$$ around a value, you have implicitly restricted your model to an $$n$$ degree polynomial and have set an initial value for $$\theta$$. A polynomial is linear in basis and thus if you use a convex loss function can be solved exactly given your data. No further expansion can improve this result.
• But then I'm estimating $\theta$, so I could produce a new Taylor expansion around that new value and re-estimate. So process is: expand around arbitrary $\theta$, estimate $\theta$, expand around new $\theta$, estimate $\theta$, etc etc – luchonacho Dec 7 '18 at 18:56
• If $g$ is a polynomial, when you estimate $\theta$ using a linear model (which you can do) you will get the correct (up to a small error) $\theta$ for which there is no improvement. – Chris Dec 7 '18 at 20:57