2
$\begingroup$

When reading and watching video regarding building and using Bayes Nets, the examples typically use binary outcomes for the nodes.

'Probability of it raining', 'Probability of x disease', ect...

With those types of nodes it is easy to derive the conditional probability table. However, what do you do when you have continuous variables with many conditional variables or a more complicated binary model?

Would you use standard regression techniques like OLS and Logistic regression to estimated the nodes of a Bayes Net?

Or what other methods might one use?

$\endgroup$
  • $\begingroup$ I would clarify what you mean by "would you use [techniques] to estimated the nodes of a Bayes Net?" Do you want to learn the structure of a bayes network given continuous variable outcomes? $\endgroup$ – Vimal Jul 13 '15 at 15:40
  • $\begingroup$ I suppose I don't know the answer to that question. I know that I am clearly missing something. I want to know how to estimate the probability of an outcome given multiple variables affecting it. I prefer the continuous case. I guess my first thought was the techniques mentioned in my question, but I believe this is a naive thought. $\endgroup$ – Michael Jul 13 '15 at 16:17
  • $\begingroup$ To clarify, you would like to know how to find the probability distribution for a node given its parents connections, correct? It sounds like your first step will be choosing the form of the probability distribution; maybe a multivariate Gaussian or some other multivariate distribution. Then, you could use a Bayesian approach to find the most likely parameters for that distribution given observed data and the observed data for the parent nodes. $\endgroup$ – Christopher Krapu Jul 13 '15 at 18:27
1
$\begingroup$

Would you use standard regression techniques like OLS and Logistic regression to estimated the nodes of a Bayes Net?

If I understand your question correctly, you would like to model $P({\rm outcome} | {\rm features})$, where features are random variables in the continuous domain and outcome is probability of 0 or 1. Assume you are given training samples $(x_i, y_i)$, where $x_i$ are your features (real valued) and $y_i$ are the outcomes (binary valued, 0 or 1).

You can model $P(Y|X)$ using a family of approaches. Logistic Regression (a single-layer neural network) basically models $P(Y|X) = sigmoid(A \cdot X + b)$, where $A, b$ are your parameters (feature weights and bias, respectively). That is, it assumes that the log-odds ($\log(p/(1-p))$) is a linear function of your features. You can read up more on https://en.wikipedia.org/wiki/Logistic_regression.

If you train the above function with a specific loss function (the cross-entropy loss, which you can read on the Wiki page), the minimiser will approximate the true posterior probability $P(Y|X)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.