3
$\begingroup$

If I fit the model mgcv::bam(y~s(x), rho=0.8), where y and x are ordered by time, my understanding is that the model can be described in math notation as:

$ y_t = \beta0+f(x_t)+\epsilon_t\\ \epsilon_t = rho * \epsilon_{t-1} + w_t\\ w_t \sim normal(0, \sigma) $

Is this correct? Also is the maximum likelihood estimate of the model equivalent to minimizing the square of $w$?

$\endgroup$

1 Answer 1

2
$\begingroup$

I tend to think of the model mgcv::bam(y ~ s(x), rho = 0.8) as being equivalent to

$$ y_t = \beta_0 + f(x_t) + \varepsilon_t $$

where $\boldsymbol{\varepsilon} \sim \mathcal{N}(0, \hat{\sigma}^2\boldsymbol{\Lambda})$ and $\boldsymbol{\Lambda})$ is the first order correlation matrix

$$ \boldsymbol{\Lambda} = \begin{pmatrix} 1 & \rho & \rho^2 & \cdots & \rho^{n-1} \\ \rho & 1 & \rho & \cdots & \rho^{n-2} \\ \rho^2 & \rho & 1 & \cdots & \rho^{n-3} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \rho^{n-1} & \rho^{n-2} & \rho^{n-3} & \cdots & 1 \\ \end{pmatrix} $$

with $\rho = 0.8$.

But this arises from the assumed first order AR process

$$\varepsilon_t = \rho \varepsilon_{t-1} + \eta_t$$

with $\eta_t \sim \mathcal{N}(0, \hat{\sigma}^2)$. (Which is what you wrote but with the notation from my own lecture notes/slides so as to not confuse myself.)

This is essentially a generalised least squares problem, which I understand will seek to minimise the squared Mahalanobis length of the residual vector $\eta_t$ (your $w_t$).

$\endgroup$
2
  • $\begingroup$ Thank you. I had read your notation, N(0,σ^2Λ), before without really understanding it. The explicit example for the AR(1) model here really helped. $\endgroup$
    – JohannesNE
    Jan 14, 2022 at 11:51
  • $\begingroup$ Gavin Simpson and David Lawrence Miller shared a lot of valuable insights regarding this question on twitter (twitter.com/JohsEnevoldsen/status/1481922275632898056) $\endgroup$
    – JohannesNE
    Jan 14, 2022 at 12:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.