I am working with a model that contains a large number of coefficients, arranged in an ordered vector $\beta_1, \dots, \, \beta_N $. I have some prior knowledge that could be used to improve the estimation of the $\beta$, in particular:
- There's autocorrelation in the vector of $\beta$, such that the difference between contiguous values, e.g. $\beta_i$ and $\beta_{i+1}$, is more likely to be 'small', rather than 'large';
- I expect most $\beta$ coefficients to be unimportant; specifically I expect only a single, relatively small (compared to $N$), cluster of contiguous coefficient to be different from zero and useful for predicting my dependent variable.
I am working in a Bayesian setting (I use Stan). My idea was to include both these information in the model using a combination of a random-walk prior (to enforce smoothness and correlation between contiguous $\beta$) with a regularizing prior, to give higher prior probability to small values of $\beta$.
More in detail, I have a random walk prior such that $$ \beta_1 \sim \mathcal{N}(0,1), \quad \beta_i \sim \mathcal{N}(\beta_{i-1}, \tau) $$ where $\tau$ is a smoothness hyperparameter.
At the same time I want to penalize large values of $\beta$. In ridge regression this would be achieved by adding a penalty $-\lambda \sum_{i=1}^N \beta_i^2$ to the log-likelihood. In a Bayesian setting this should be equivalent to setting a zero-centred Gaussian prior, e.g. $$ \beta_i \sim \mathcal{N}\left(0, \frac{1}{2\lambda}\right) $$
Putting everything together, this would give the following modified/regularized random walk prior: $$ \beta_1 \sim \mathcal{N}\left(0, \: \frac{1}{1+2\lambda}\right), \quad \beta_i \sim \mathcal{N}\left(\frac{\frac{1}{2\lambda}}{\tau + \frac{1}{2\lambda}}\beta_{i-1}, \: \frac{1}{\frac{1}{\tau} + 2\lambda} \right) $$
My question is: does this approach makes sense or are there shortcomings/limitations that I have missed? Is there any reference I can cite that used similar approach? Or alternatively, is there a better way to do this?
Thanks!
