Timeline for Why should I be Bayesian when my dataset is large?
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 20, 2020 at 13:34 | comment | added | Tim | @jpcgandre when the data grows larger, the impact of priors diminishes. You can use Bayesian methods with large datasets. For many problems you need uncertainty estimates, Bayes methods are one of the possible solutions. | |
| Nov 20, 2020 at 13:22 | comment | added | jpcgandre | @Tim I agree that it gives you uncertainty estimates, but still uncertainty estimates around a biased posterior. | |
| Nov 20, 2020 at 13:16 | comment | added | Tim | @jpcgandre Bayesian neural networks were used in here just as an example, Bayesian models are not restricted to neural networks as well. The key point is that Bayesian approach gives you uncertainty estimates, having big data does nothing about giving you uncertainty estimates. | |
| Nov 20, 2020 at 12:55 | comment | added | jpcgandre | @Tim Machine learning is not restricted to neural networks. There are several methods which do not require using a bayesian framework. What makes me rethink of using bayes in ML and when large data is the variance-bias trade-off. Bayesian methods tend to reduce variance but not bias, whereas data driven methods usually reduce bias but have larger scatter. Big data makes it possible to reduce the scatter in the latter, not sure the bias in the former. | |
| Oct 7, 2020 at 8:51 | comment | added | Tim | @kennysong if your data is small, then the model can memorize the data (overfitt) and give exact predictions per each datapoint. In such case it would "feel" certain about it's predictions. If the data is big, than it cannot be certain, because same parameters need to be re-used for different data, so there is a variability. | |
| Oct 7, 2020 at 3:41 | comment | added | kennysong | @JeremyList Why is that the case? It seems counterintuitive to me | |
| Oct 7, 2020 at 1:23 | comment | added | Jeremy List | @kennysong BNNs do collapse to certainty for small datasets; I think larger datasets make that type of collapse less probable rather than necessarily prevent it. | |
| Oct 6, 2020 at 23:12 | comment | added | kennysong | Is there some intuition of why the uncertainty estimates of BNNs don't collapse to certainty for large datasets? (Maybe they do?) | |
| Oct 6, 2020 at 21:33 | history | answered | Tim | CC BY-SA 4.0 |