8
$\begingroup$

Wikipedia defines it like this:

The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs given inputs that it has not encountered.

Can this be seen as a prior in the Bayesian framework, and if not, why?

$\endgroup$
1
  • $\begingroup$ I think so, or that a Bayesian prior is a kind of inductive bias. It tells the analyst how to generalize the known to the unknown, with conclusions being "biased" towards things that were taken as likely a priori. $\endgroup$
    – dsaxton
    Commented Dec 17, 2015 at 21:31

3 Answers 3

1
$\begingroup$

A prior is a property of the data and not the algorithm used on the data.

"Maximum conditional independence: if the hypothesis can be cast in a Bayesian framework, try to maximize conditional independence. This is the [inductive] bias used in the Naive Bayes classifier." - Wikipedia

Inductive biases can be thought of as the assumptions about the data encoded in the learning algorithm.

$\endgroup$
1
$\begingroup$

Prior is a prior knowledge that can help us learn new concepts from the available data.

Prior or inductive prior is also known as inductive bias.

In here, the word inductive does not hold the strict mathematical meaning of induction, but rather the fact that we will make some inference based on the previous knowledge.

In this sense inductive bias is a prior (prior distribution) which is the knowledge about the data (without observing any data).

The posterior distribution is a knowledge after the new evidence (the data) has been observed, taking in account the prior knowledge.

In this sense you may guess the distribution is the knowledge about the data.

$\endgroup$
0
$\begingroup$

No. This is the likelihood as evident from the defintion. You introduce inductive bias in linear regression by making assumption that the data follows linear model. Even in Bayesian approaches you're introducing some inductive bias by assuming that data follows a model from a certain family.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.