Situation/Data: I'd like to start with an example from climate science. Suppose you have a univariate time series $\vec{z} = (z_1, z_2, ..., z_n)^T$, where $z_t$ are block maxima of time step $t\in1,..,n$ (e.g. annual precipitation maxima at a specific station). Additionally, one has a large amount of (strongly correlated) predictor variables (e.g. surface pressure at $s$ different weather model grid points), such that $\pmb{X} = \big(\vec{x}_{\text{loc1}}, ..., \vec{x}_{\text{loc}s}\big)^T \in {\rm I\!R}^{n \times s}$ is the predictor matrix (c.f. Figure 1).
Figure 1: Block maxima time series $\vec{z}$ (Top row) and three predictor time series
Model: Since $\vec{z}$ are block maxima, we model them with a non-stationary Generalised Extreme Value GEV distribution $G(z;\mu(\vec{x}_t),\sigma,\xi)$,
$G(z_t;\mu(\vec{x}_t),\sigma,\xi) = \text{exp}\Big\{-\big[1+\xi \frac{z-\mu(\vec{x}_t)}{\sigma}\big]^{-1/\xi}\Big\}$ (1)
where the location parameter $\mu_t$ - in contrast to the scale parameter $\sigma$ and the shape parameter $\xi$ - depend linearly on our predictors $\vec{x}_t$ at time point $t$
$\mu_t(\vec{x}_t) = \mu_0 + \mu_1 x_{\text{loc1},t} + \mu_1 x_{\text{loc2},t} + ... + \mu_s x_{\text{loc}s,t}$. (2)
To estimate the respective parameters $\vec{\theta} = (\mu_0, \mu_1, ..., \mu_s, \sigma, \xi)$, one optimises the negative log-likelihood (Coles, 2001)
$\ell(\mu(\vec{x}_t),\sigma,\xi) = - \sum_{t=1}^{n}\Big\{\text{log}\,\sigma + (1+1/\xi) \text{log} \big[1+\xi\big(\frac{z_t-\mu(\vec{x}_t)}{\sigma}\big)\big] + \big[1 + \xi \big(\frac{z_t-\mu(\vec{x}_t)}{\sigma}\big) \big]^{-1/\xi}\Big\}$ (3)
i.e. $\widehat{\vec{\theta}} = \text{argmax}_{\vec{\theta}} \ell(\mu(\vec{x}_t),\sigma,\xi)$.
Model comparison or predictor selection would be done using a $\chi^2$ test on the deviance statistic $D=2\{\ell_1(\mathcal{M}_1)-\ell_0(\mathcal{M}_0)\}$, where $\mathcal{M}_0 \subset \mathcal{M}_1$, i.e. the larger model $\mathcal{M}_1$ is provided with more predictors than $\mathcal{M}_0$.
Issue: However, the predictor time series stem from a spatial field, thus they are strongly correlated. To account for the (spatial) correlation amongst predictors, the idea was to regularise the negative log-likelihood using elastic net (or just ridge/lasso regularisation), e.g. $l_{\text{reg}}(\mu(\vec{x}_t),\sigma,\xi) = l(\mu(\vec{x}_t),\sigma,\xi) - \lambda \sum_{i=1}^{s} \mu_i^2$ for ridge (Hastie et al., 2009).
Questions:
- In general: Is it meaningful to apply regularisation to the negative log-likelihood?
- If we would not regularise, would we have to account for the correlation in the predictors when using a $\chi^2$ test or AIC/BIC?
- Could one still use a $\chi^2$ test to compare models which were regularised (i.e. $\ell_{\text{reg}} \neq \ell$)?
- Getting slightly philosophical: (How) would it be possible to combine the flexibility of statistical learning methods with the solid foundations of extreme value theory? Chapter 2.4 of (Hastie et al., 2009) summarises statistical theory required to find optimal models for prediction, e.g. $f(x) = {\rm I\!E}(Y|X=x)$ for a squared error loss, providing the ground for a large amount of statistical learning methods. Is there a way to integrate this large class of models (for example using Generalised Additive Models to model the location parameter $\mu(\vec{x})$ in Equation 2) into a framework where the distribution of $Y$ is known from theory (e.g. the GEV in extreme value theory).
Coles, S. (2001). An introduction to statistical modeling of extreme values. Springer series in statistics. London: Springer, London, 3rd print edition.
Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning. Springer Seriesin Statistics. Springer New York, New York, NY.
