# Sequential problem for n=1, non linear regression

I am trying to understand an example in my stats course notes, the example relates to calculating the best value for the next experiment.

The function of the line is very simple:

$$\ln(Y_i) = \ln(\theta^*_1 + \theta^*_2x_i) + \epsilon_i$$

The example works on a previous example where point estimates have already been obtained for $\theta^*_1$ and $\theta^*_2$ These are:

$$\theta = \begin{pmatrix} 3.9963\\ 2.3792 \end{pmatrix}$$

i     y       x
1     4.11    0
2     6.32    1
3     8.21    2
4     10.43   3
5     14.29   4
6     16.78   5


So the question is, how to get the optimal value for $x_7$.

The equation that needs to be used is given, but doesn't make a lot of sense to me, particularly the last line, where some seemingly random numbers appear from nowhere.

Here are the notes provided:

Let the $p\times 1$ vector $$r_{n+1} = \frac{\delta f(x_{n+1}. \theta)}{\delta\theta}$$

$$C_{n+1} = X_{n+1}'X_{n+1} = \begin{pmatrix}X_n\\ r_{n+1}' \end{pmatrix}' \begin{pmatrix}X_n\\ r_{n+1}' \end{pmatrix}$$

$$\phi = |C_{n+1}| = |\begin{pmatrix}X_n\\ r_{n+1}' \end{pmatrix}' \begin{pmatrix}X_n\\ r_{n+1}' \end{pmatrix}| = |X_n'X_n+r_{n+1}r_{n+1}'|$$ $$\therefore \phi = |C_{n+1} + r_{n+1}r_{n+1}'| = |C_n|(1 + r_{n+1}'C_n^{-1}r_{n+1})$$

Here comes the example "at which value of $x, 0 \leq x \leq 2$ should the next experiment be carried out"

$$\ln(Y_i) = \ln(\theta^*_1 + \theta^*_2x_i) + \epsilon_i$$

$$\theta = \begin{pmatrix} 3.9963\\ 2.3792 \end{pmatrix}$$

$$X'X = \begin{bmatrix}0.11777 & 0.11660\\ 0.11660 & 0.33500 \end{bmatrix}$$

$$r_7' = \begin{bmatrix} \frac{1}{\theta_1 + \theta_2 x_7} & \frac{x_7}{\theta_1 + \theta_2 x_7}\end{bmatrix}$$

Now, while I don't exactly understand all this - I can work out what its saying. What gets me is the next bit.

$$\phi = \frac{12.9358 - 8.9781x_7 + 4.534x_7^2}{(3.9963 + 2.3792x_7)^2}$$

Where did 12.9358, 8.9781, 4.534 come from?

Also, where did the division part of this come from, above $\phi$ was calculated without any division at all.

Finally, where did the $^2$ come from on the divisor?

Any help here would be greatly appreciated, I get the feeling its going to be very simple - I'm just missing something.

Thanks

EDIT After much deliberation, I have figured out this:

$$\phi = \frac{X'X^{-1}_{00} + (X'X^{-1}_{0,1} + X'X^{-1}_{1,0})x_7 + X'X^{-1}_{1,1}x_7^2}{(\theta_0 + \theta_1x_7)^2}$$

It would be really fantastic if someone could confirm that this is the case for ALL non linear regressions, or if this is only the case for this one, and if it is something different for other equations.

• what does the prime (') mean here? is that the derivative? Commented Apr 10, 2013 at 22:00
• transpose of the matrix. Commented Apr 10, 2013 at 23:09
• and |X| is the determinant of X? Commented Apr 10, 2013 at 23:57
• That is correct Commented Apr 11, 2013 at 0:10
• When you say 'optimal', what is to be optimized? Is $\phi$ to be minimized? Commented Apr 11, 2013 at 7:46

$|X'X+r_7r_7'|=| \begin{bmatrix}0.11777 & 0.11660\\ 0.11660 & 0.33500 \end{bmatrix} + \begin{bmatrix} \frac{1}{\theta_1 + \theta_2 x_7}\\ \frac{x_7}{\theta_1 + \theta_2 x_7}\end{bmatrix} \begin{bmatrix} \frac{1}{\theta_1 + \theta_2 x_7} & \frac{x_7}{\theta_1 + \theta_2 x_7}\end{bmatrix}|$

• I'm sorry, but where do you see X'X used? its always $X_n'X_n$, and in this case n = 6 and n+1 = 7, because we are trying to determine which value of x the 7th parameter should use. Also, my question is mainly how did we get 12.9358, 8.9781 and 4.534, your answer wouldn't result in any of those numbers would it? And certainly not all 3 of them Commented Apr 11, 2013 at 0:09
• So I don't understand the notation, what is $X$? is that $X_0$? Commented Apr 11, 2013 at 0:17
• Where do you see X alone, without an n or a 7? Its a bit confusing isnt it. If it helps any, I happen to know that when you make $x_7=0$ $\phi=0.8$, thats what the equation at the bottom gives, my primary question is how do we get from all the definitions of equations, to the one at the bottom? Commented Apr 11, 2013 at 0:19
• I see $X'X$, how does this relate to the rest of the problem? and while we are at it, what does the dot triangle mean? Commented Apr 11, 2013 at 0:21
• Excellent question...All I know is that what is listed above, is exactly what is given in the example. Perhaps $X_n'X_n$ is referencing the $X'X$ matrix, rather that the matrix $X$ matrix. But there are only 2 rows in that, wheras there are 6 rows in the $X$ matrix. However, $r_n$ references this matrix, or at least adds a row to it, as the $X$ matrix was gotten from this same formula Commented Apr 11, 2013 at 0:25

Either the formula for $\phi$ or the answer is wrong. The answer $\phi$ is actually the term $r_{n+1}'C_{n}^{-1}r_{n+1}$. The constants you are puzzled about come from the inverse of $X'X$.

$$(X'X)^{-1}=\begin{bmatrix} 12.955677 & -4.509349\\ -4.509349 & 4.554597 \end{bmatrix}$$

The formula for $\phi$ then comes from the observation that $r_n'=1/(3.9963+2.3792x_7)[1,x_7]$.