# Bias correction when using loo cross-validation to replace unreliable PSIS-LOO estimates

The PSIS-LOO information criterion (see this paper by Vehtari, Gelman, and Gabry) assigns a Pareto shape parameter $$\hat k$$ to each observation in the data, and these $$\hat k$$ values can be used to assess the reliability of the estimates.

According to the authors:

Even if the PSIS estimate has a finite variance, when $$\hat{k} > 0.7$$ the user should consider sampling directly from $$p(\theta^s |y_{−i})$$ for the problematic $$i$$, use K-fold cross-validation, or use a more robust model.

The first option, sampling directly from $$p(\theta^s |y_{−i})$$ for the problematic $$i$$, has been implemented in the R packages rstanarm and brms.

This earlier paper by Gelman, Hwang, and Vehtari states that, to correct for the left-one-out fits being conditioned on one less observation, one can add a first-order bias correction $$b$$ to the loo cross validation estimate (see section 3.8).

Main question: How should we include $$b$$ when replacing bad PSIS-LOO estimates with explicit loo cross validations?

In the notation of the Gelman, Hwang, and Vehtari paper, rstanarm and brms both replace a bad PSIS-LOO estimate for row $$i$$ with $$\log p_{\text{post}(-i)}(y_i)$$. Based on the definition for $$b$$, it seems like the bias-corrected version of this could be something like $$\log p_{\text{post}(-i)}(y_i) + b_i$$, where

$$b_i = \log p_{\text{post}}(y_i) - \frac{1}{n} \sum_{j=1}^{n} \log p_{\text{post}(-i)}(y_j).$$

Does this seem right? In tests, adding this bias correction seems to always increase the resulting looic estimate, but I'm not sure if this is just a quirk of the particular data I'm using.