In Bayesian learning, priors play an important role. As per my understanding, most of the priors are statistical, and they used mean and variance value as prior. Then deep learning model uses the dataset and creates posterior distribution. Is it possible to use symbolic knowledge, such as the first-order logic (FOL) rule, as priors? As these rules are not statistical by default. If yes, what could be the possible way to start?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
No. Bayesian deep learning is about defining neural networks as probabilistic models and using prior distributions for the parameters. This is a probabilistic model, so whatever prior knowledge you want to include, it needs to take the form of a probability distribution. It is Bayesian because it employs Bayes theorem to combine the learnings from the data (likelihood) with the priors. So "prior" has a very specific sense here.
I'm not an expert in this area, but there are symbolic machine learning models and active research in this area. I didn't read those papers carefully, but a quick search for "Bayesian symbolic machine learning" reveals at least a few papers like https://arxiv.org/abs/2211.15860 or https://arxiv.org/abs/1910.08892 that use Bayesian approach for symbolic learning, but they are using the priors in the regular sense. So they can be combined, but symbolic knowledge itself cannot "be" the prior in a Bayesian model.
-
$\begingroup$ Thanks for your insight, and you also confirmed my assumption that priors and symbolic roles might not go so well together. I assume we cannot use symbolic knowledge as priors unless we do some mathematical intermediate representation and then use them as priors. $\endgroup$ Commented Jun 28, 2023 at 9:48
-
1$\begingroup$ @Rambo_john exactly as you said. If you somehow included the knowledge in the model as some kind of "prior" it wouldn't be a "prior" in a Bayesian sense, hence it wouldn't be a Bayesian model. $\endgroup$– TimCommented Jun 28, 2023 at 9:55