I am performing linear regression using the equation $x^{*}=(A^TA)^{-1}A^Tb$, from:


I am trying to model an exponential relationship (which estimates a target, y, with A and β) of the form $y=Ae^{βt}$, which is equal to $ln(y) = α+βt$, where $A = e^{α}$

I have a target set $b$ which I took the natural log of all elements to transform it to $ln(b)$, and a matrix $A$ which has coefficients of 1 and k, like so: $A=\begin{bmatrix} 1 & k_1\\ 1 & k_2\\ 1 & k_3 \\ \end{bmatrix}$ and so on. Thus, when I plug this matrix $A$ into the equation $x=(A^TA)^{-1}A^Tb$, I expect the matrix $x$ to have coefficients $α$ and $β$ such that $ln(b) = α+βt$, which I know is a smooth function.

However, when I plot the product $Ax^{*}$ (which I think is the predicted values of $ln(b)$ I do not get a smooth function, instead I get:

Predicted b updated

Unfortunately, I have no clue what equation this represents; it does not match the graph of $ln(\hat{b}) = -4.40680302+0.00149165363t$?

I have also attached my predicted $x$ and $b$ matrices for reference.

predicted $x=\begin{bmatrix} -4.40680302\\ .00149165363\\ \end{bmatrix}$

$b = \begin{bmatrix} 0.01081389\\ 0.01918884\\ 0.01299735\\ 0.01767956\\ 0.01230499\\ 0.02024096 \end{bmatrix}$

Below is the Python code I used to perform the described computations:

weekly_pos = np.array([39,71,125,279,149,198]).reshape(-1,1)
A = np.concatenate((np.ones((len(weekly_pos),1)),weekly_pos), axis=1)
b = np.log(np.array([19/1757,44/2293,49/3770,64/3620,56/4551,84/4150])).reshape(-1,1)
u = np.linalg.inv(np.matmul(A.T,A))
v = np.matmul(A.T,b)
pred_x = np.matmul(u,v)
y_hat = np.matmul(A,pred_x)
  • $\begingroup$ Can you post a minimal working example of the code you used to produce this? $\endgroup$ – Demetri Pananos Oct 18 '20 at 22:50
  • $\begingroup$ @DemetriPananos Sure, I've just added it $\endgroup$ – H. Khan Oct 18 '20 at 22:57
  • $\begingroup$ What is the relationship between, $x$, $b$ and $y$? It seems you use $x$ in two different ways, and perhaps that $y=b$ or $\log(y)=b$ or something. This would be more clear if you used each symbol in one way only. Also, your plot shows values plotted on the horizontal axis at 0, 1, 2, ... but these are not values in $b$. So is this just a problem with how you've plotted the data? $\endgroup$ – Sycorax Oct 18 '20 at 23:18
  • $\begingroup$ The equation $\ln(\hat{b}) = .0124891386+.0000212436105x$ does not make sense if $x$ is the estimated coefficients of the linear model. $\endgroup$ – Sycorax Oct 18 '20 at 23:27
  • $\begingroup$ @Sycorax That's sort of what I'm trying to figure out - what equation correctly relates those two values in my x* matrix? $\endgroup$ – H. Khan Oct 18 '20 at 23:29

After the edits, I think the only remaining source of confusion is that is the prediction on the log scale, so instead we can write $$ \begin{align} \ln(\hat{b}) &= -4.40680302+0.00149165363 t \\ \hat{b}&=\exp(-4.40680302+0.00149165363t) \\ &= \exp(-4.40680302)\exp(0.00149165363t) \\ &= \hat{A}\exp(\hat{\beta} t) \end{align} $$

As for your plot, the problem is that

  • on the horizontal axis, you're not plotting $t$, but the indices of $t$, i.e. $[0,1,2,\dots,5]$. Instead, use your values for $t$ (which are called weekly_pos in your code). This is important because weekly_pos is not sorted, so you get the zig-zag pattern as a result of using the indices.
  • on the vertical axis, you're plotting $\ln(\hat{b})$, not $b$. So you should exponentiate your predictor if you want predictions on the linear scale.

For some information about the bias of this procedure, see Predicting y from log y as the dependent variable

  • $\begingroup$ It might be worth pointing out that this approach yields predictions that are biased low. $\endgroup$ – whuber Oct 19 '20 at 13:17
  • $\begingroup$ @whuber This is a good suggestion. I've tried to pull a good Question about this into the post but if you have a different suggestion, I'll happily make the edit. $\endgroup$ – Sycorax Oct 19 '20 at 16:06
  • $\begingroup$ What I would be most concerned about is the contradiction between the two models proposed by the OP: it is implied that each has additive error, but that's not possible. $\endgroup$ – whuber Oct 19 '20 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.