After searching Wikipedia, I found that there are both parametric Bayesian models and non-parametric Bayesian models. What about Bayesian Networks? When building up a Bayesian Network model, I don't need to give much prior information. Does that mean Bayesian Networks are non-parametric models?
2
-
2$\begingroup$ "I don't need to give much prior information" Ironically this is the exact distinction you are missing. On what do you set your priors? A finite set of path weights and residual variances? You're looking parametric. On an infinite space of probability models? You're veering into non-parametric. $\endgroup$– AdamOApr 28, 2021 at 15:38
-
1$\begingroup$ No. Bayes nets can be parametric. It only has to do with the models used to relate edges. Non-parametric Bayesian regression models to estimate paths in the graphical model make the Bayesnet a non-parametric Bayes net. If you just use linear regression (inverse gaussian prior on residual variance for regressions between nodes), it's a parametric Bayesnet procedure. $\endgroup$– AdamOApr 28, 2021 at 18:11
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Based on those definitions:
Nonparametric:Algorithms that do not make strong assumptions about the form of the mapping function are called nonparametric machine learning algorithms. By not making assumptions, they are free to learn any functional form from the training data.
EX: k-Nearest Neighbors, Decision Trees
Parametric:Assumptions can greatly simplify the learning process, but can also limit what can be learned. Algorithms that simplify the function to a known form are called parametric machine learning algorithms.
EX: Logistic Regression, Linear Discriminant Analysis
And in my knowledge I can: Yes, Bayesian Belief Networks with discrete variables are indeed nonparametric, because they are probabilistic models based conditional dependencies between their variables. I worked with those models (discrete) in ecology.
BUT,Bayesian networks with continuous variables make strong assumptions about their data (I believe one of them is that the variables follow a Gaussian distribution). These models themselves are parametric.
-
2$\begingroup$ Where are these definitions coming from? I don't think these are accurate taken either from stat or ML literature. $\endgroup$– AdamOApr 28, 2021 at 15:34