In bias and variance decomposition theorem, bias and variance are all related to distribution, which means they should be related to a group of models. But people always say this model is high bias or high variance, which makes me very confused. Kindly share me some thoughts!
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Just one model. Defining its complete distribution is generally very hard if not impossible, and even its domain space can be hard to figure, mostly in the case of non parametric models like many typically used in ML, but since training data is random, the model trained on it will be random too.
You may find useful this Q&A.
In the end, what you are interested in is just bias and variance, and, more often than not, just in a theoretical way: you don't have to really measure them, you just have to understand that some models are too sensible to data fluctuations (high variance models), while others are stuck too far from the target (high bias models). The target here is the "perfect model", that predicts everything as well as possible, and you need a bit of bias and a bit of variance to stay as close as possible to it.
By the way, this also applies to unsupervised models were there are no predictions.
-
$\begingroup$ Thx, Carlo. It is a good explanation. $\endgroup$ Commented Apr 23, 2020 at 2:03
-
$\begingroup$ you are welcome. you can accept the answer then $\endgroup$– carloCommented Apr 23, 2020 at 10:02
-
$\begingroup$ Yes, absolutely. That green tick, right? I clicked it. $\endgroup$ Commented Apr 23, 2020 at 16:31
-
$\begingroup$ yeah, thank you. I hate things left halfway $\endgroup$– carloCommented Apr 23, 2020 at 20:38