Skip to main content
edited tags
Link
gung - Reinstate Monica
  • 147.5k
  • 89
  • 406
  • 717
edited title
Link
Ébe Isaac
  • 1.1k
  • 1
  • 18
  • 27

Examples How will one determine a classifier to be of high bias andor high variance classifiers?

Tweeted twitter.com/StackStats/status/968520711571759106
deleted 1 character in body
Source Link
Ébe Isaac
  • 1.1k
  • 1
  • 18
  • 27

The bias and variance of a classifier determines the degree to which it can underfit and overfit the data respectively. How could one determine a classifier to be characterized as high bias or high variance?

I am pretty clear of what is a bias-variance trade-off and its decomposition and how it could depend on the training data and the model. For instance, if the data does not contain sufficient information relating to the target function (to simply put it, lack of samples), then the classifier would experience high bias due to the possible incorrect assumptions it would make. On the contrary, if the classifier tightly fits the given training data (say, an ANN with a lot of nodes running multiple epochs or ana decision tree with a high depth), then it would exhibit high variance because it cannot generalize well to predict unseen samples.

However, there are cases where I see lectures talking about selecting a high bias low variance classifier, or a low bias high variance classifier. E.g., naive Bayes is considered to be a high bias low variance classifier (I presume it is due to the conditional independence assumption). How to determine this? So how will one characterize SVM, ID3, Random Forests, and $k$NN? Are they high bias or high variance?

The bias and variance of a classifier determines the degree to which it can underfit and overfit the data respectively. How could one determine a classifier to be characterized as high bias or high variance?

I am pretty clear of what is a bias-variance trade-off and its decomposition and how it could depend on the training data and the model. For instance, if the data does not contain sufficient information relating to the target function (to simply put it, lack of samples), then the classifier would experience high bias due to the possible incorrect assumptions it would make. On the contrary, if the classifier tightly fits the given training data (say, an ANN with a lot of nodes running multiple epochs or an decision tree with a high depth), then it would exhibit high variance because it cannot generalize well to predict unseen samples.

However, there are cases where I see lectures talking about selecting a high bias low variance classifier, or a low bias high variance classifier. E.g., naive Bayes is considered to be a high bias low variance classifier (I presume it is due to the conditional independence assumption). How to determine this? So how will one characterize SVM, ID3, Random Forests, and $k$NN? Are they high bias or high variance?

The bias and variance of a classifier determines the degree to which it can underfit and overfit the data respectively. How could one determine a classifier to be characterized as high bias or high variance?

I am pretty clear of what is a bias-variance trade-off and its decomposition and how it could depend on the training data and the model. For instance, if the data does not contain sufficient information relating to the target function (to simply put it, lack of samples), then the classifier would experience high bias due to the possible incorrect assumptions it would make. On the contrary, if the classifier tightly fits the given training data (say, an ANN with a lot of nodes running multiple epochs or a decision tree with a high depth), then it would exhibit high variance because it cannot generalize well to predict unseen samples.

However, there are cases where I see lectures talking about selecting a high bias low variance classifier, or a low bias high variance classifier. E.g., naive Bayes is considered to be a high bias low variance classifier (I presume it is due to the conditional independence assumption). How to determine this? So how will one characterize SVM, ID3, Random Forests, and $k$NN? Are they high bias or high variance?

Source Link
Ébe Isaac
  • 1.1k
  • 1
  • 18
  • 27
Loading