1
$\begingroup$

I am sampling a beta using a Gibbs sampling. It is variable selection model. So in different iteration of gibbs different covariates are included to the model (denoted by a variable selection indicator $\gamma$). However, this indicator have high variance ( this is because my covariates are correlated). I use: $\hat\beta = 1/n\sum \beta_{samples} $ as my estimate for further analysis. This turn out to be problematic because if I calculate the explained variance it is negative. On the other hand if I just use a single sample of $\beta_{sample}$ to find explained variance it is correct. What's wrong here?

Do I need to re-estimate $\beta$ for nonzero mean $\gamma = 1/n \sum \gamma_{sample}$? This does not sound correct because this estimate does not correspond to mean of posterior of $\beta$. thanks

$\endgroup$
0
$\begingroup$

I found on thing that helps to reduce the variance in $\gamma$ was to use Rao-blackwelization. Its just taking the expected the posterior of $\gamma$ as final extimate i.e. $\hat\gamma = 1/n\sum_i p_{samples}(\gamma| Data, other-params)$. If I take only those $\beta$ which $\hat \gamma >> 0 $, it removes the problem of negative explained variance. However, I think still it is not correct samples from posterior of $\beta$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.