2
votes
0answers
50 views

Mixture of probits: understanding truncated-based likelihoods

I am trying to implement a mixture model of probits to infer the best decision boundary for every latent subpopulation. When doing Gibbs sampling, we eventually have to compute $P(y^* | w_c)$ where ...
1
vote
0answers
22 views

Bayesian Perceptron - how can I generate many different perceptrons?

I am going to implement the Bayesian version of a perceptron that I read in the Statistical Mechanics of learning, by Engel-Van Den Broeck. The idea to improve the performance is to use many Gibbs ...
3
votes
0answers
46 views

Stochastic Programming (e.g. LP) with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
0
votes
0answers
39 views

What is the Probability Distribution of NLTK Naive Bayes?

As I know Naïve Bayes has various distributions, as said in Sci-kit learn manual “The different naive Bayes classifiers differ mainly by the assumptions they make regarding the distribution of P(x_i ...
0
votes
1answer
39 views

What is the difference between Binary Clasification and Multiclass classification?

Apology for posting almost one question daily. I am trying to learn some aspects of Statistical Machine learning, so every day many questions coming and if I am not finding answer in my offline peer ...
0
votes
0answers
26 views

What machine learning tool is best suited for taking time series data as well as descriptive data and making a binomial classification

I have an interesting task of utilizing log data from computer servers in a server farm and predicting if a particular server is likely to fail in the next 24 hours. My data set will be comprised of ...
1
vote
1answer
43 views

Estimation in Naive Bayes

I have a very silly question. In Multinomial Naive Bayes Classifier, which parameter estimation do we use, is it Maximum Likelihood or Maximum A Posteriori? If any one of the esteemed members may ...
0
votes
0answers
33 views

What does it take to learn a Model?

In Machine Learning, there are various models. I tried to learn few probabilistic models of Machine Learning. I read the theory, worked on problems and tested my results and could analyze my data ...
2
votes
1answer
54 views

How to construct a reasonable prior and likelihood for Bayes modelling?

To apply Bayes inference for data analysis or machine learning, we have to construct prior and likelihood, right? But if I fail to come up with a reasonable prior and likelihood, then the Bayes model ...
0
votes
1answer
113 views

Intuitive explanation of Bayesian logistic regression?

I'm looking for an intuitive explanation of Bayesian Logistic Regression (I'm using it for texts if that's relevant). It seems that this article presents it, but it's, uh, way too mathy. Thanks!
0
votes
1answer
43 views

What are the general strategies in creating a Probabilistic Graphical Model?

While there is lot of theory and probability in the background to understand, I wanted to know if there are any resources/quick pointers on what to consider while modeling a problem using Bayesian ...
1
vote
0answers
53 views

difference between hierarchical dirichlet process and nested dirichlet process

There have some extensions to Dirichlet process. One is Hierarchical Dirichlet process, and another is Nested Dirichlet Process. What are the differences between these two? I once read the paper of ...
-2
votes
1answer
95 views

ML vs MAP estimation, when to use which?

ML = Maximum Liklihood MAP = Maximum a-posteriori ML is intuitive/naive in that it starts only with the probability of observation given the parameter (i.e. the likelihood function) and tries to ...
1
vote
1answer
73 views

understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for ...
3
votes
0answers
42 views

What is the posterior probability of the data given the model used for model averaging with Bayesian Logistic Regression?

I am trying to learn about Bayesian Model Averaging using Bayesian Logistic Regression (Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text ...
4
votes
1answer
182 views

Bayesian MLPs using the MCMC methods - any tricks of the trade?

Having used the NETLAB library for MATLAB to implement Bayesian Multi-Layer Perceptron (MLP) neural networks using MacKay's evidence framework, I am now experimenting with Markov Chain Monte Carlo ...
3
votes
0answers
46 views

Reconstruct a “blocky” picture?

Consider a finite set $A$. Let the sample space be $A\times A$. We have an unknown probability distribution $f$ on this sample space. Now this probability distribution has a "blocky" property, which I ...
0
votes
1answer
69 views

Sufficient number of sample to learn Bayesian network?

I want to construct Bayesian network for a 800 genes(genes are my node/variables). I have only 30 cancer samples and 30 normal sample.so I want to create network for cancer samples and for the normal ...
2
votes
1answer
118 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
3
votes
2answers
171 views

What is the relationship between graphical models and hierarchical Bayesian models?

I've searched a good bunch of literature but have failed to find an exact distinction between the two. My impression is that in the Machine Learning literature you'll find allusions to hierarchical ...
0
votes
2answers
107 views

Direct Sampling from posterior distribution

Why is direct sampling from the posterior distribution intractable?
0
votes
0answers
45 views

Intuitive understanding of Local Probability Distribution

I'm learning Bayesian network. I have problem in intuitive understanding of Local Probability Distribution. Can anybody explain to me what it is?
1
vote
0answers
65 views

Variational Posterior Dirichlets in LDA

I am running the c code for LDA provided on David Blei's website. The code outputs several files. The output file final.gamma is supposed to include the "Variational Posterior Dirichlets". If I ...
2
votes
0answers
161 views

Good libraries for working with probabilistic graphical models?

Could someone recommend some well-maintained and up-to-date libraries for working with probabilistic graphical models? I noticed that there are some libraries for R listed here and one for C++, but ...
8
votes
1answer
128 views

regarding conditional independence and its graphical representation

When studying covariance selection, I once read the following example. With respect to the following model: Its covariance and inverse covariance matrix are given as follows, I do not understand ...
1
vote
0answers
339 views

How to compute the maximum a posteriori probability (MAP) estimate with / without a prior

I am a newbie in this area so I hope someone could explain the following problem to me in plain English. Assume I want to use MAP to estimate some parameters on the basis of some observations. I know ...
0
votes
1answer
184 views

Estimating parameters in multivariate classification resulting zero determinant sample covariance matrix

Newbie here typesetting my question, so excuse me if this don't work. I am trying to give a bayesian classifier for a multivariate classification problem where input is assumed to have multivariate ...
1
vote
1answer
186 views

Are posterior probabilities from a Naive Bayes classifier reliable?

I have read that the posterior probabilities of Naive Bayes classifiers are unreliable. Is this true? and if so, in what sense, and why? Specifically, I am interested to know if the probabilities can ...
4
votes
0answers
108 views

What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
0
votes
0answers
50 views

ML: A generalized human voting/recommendation system

I'd like to create a certain kind of voting/recommendation system. I'm sure there must be a name for what I'm trying to do, but I'm not sure. Basically, I start with a distribution of binary vectors ...
2
votes
1answer
92 views

The meaning of convergence in Variational Inference?

My friend and I are discussing about the convergence of Variational Inference, especial for Expectation Propagation method. After running some loops, the likelihood of my graphical model can be ...
1
vote
1answer
122 views

Why doesn't ML point estimate equal MAP point estimate even though I'm using uniform prior?

I asked a previous question about why the ML and MAP estimates are the same when using a uniform prior (How does a uniform prior lead to the same estimates from maximum likelihood and mode of ...
1
vote
1answer
339 views

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
0
votes
1answer
78 views

Trouble reading multinomial naive bayes notation

$C_{MAP}$: most likely class (i.e., "maximum a posteriori") $C_{NB}$: Naive Bayes x: document d is represented as $x_n$ ...
1
vote
0answers
50 views

How do I incorporate personalization to a Bayesian ranking engine?

I'm looking to quickly get smart on how to add personalization into a Bayesian-based recommendation system. I'm using clickstream data and Bayesian statistics to estimate probabilities of purchase ...
3
votes
1answer
278 views

Neural Network Black Box Workarounds

I am dealing with a data set that includes rich textual data (e.g., blog entries, magazine articles, essays, book reviews, etc.) as well as a host of proprietary metrics, including numerous ...
10
votes
2answers
1k views

Next steps after “Bayesian Reasoning and Machine Learning”

I'm currently going through "Bayesian Reasoning and Machine Learning" by David Barber and it is an extremely well written and engaging book for learning the fundamentals. So a question to someone who ...
3
votes
1answer
66 views

Learning parameters of non-parametric Bayesian models

I have a sample of Chinese restaurant process which I want to model as Pitman–Yor process. How do I determine parameters of Pitman-Yor model from given sample? For Dirichlet process I would just use ...
1
vote
3answers
107 views

How can I use Bayes rule for this question given additional data

I am required to use the Naive Bayes classifier to classify example 8, to see whether it is poisonous or not. I gained the following results: p(x|Poisonous=Y) = 0.0267857 and p(x|Poisonous=N) = ...
0
votes
0answers
114 views

Understanding the derivation of an equation in LDA modeling

When reading the derivation of LDA models, I usually get the following equations. I do not quite understand the second step, where $p(\mathbf{z}_{-i},\mathbf{w}|\alpha,\beta)$ was removed. Is that ...
1
vote
0answers
91 views

How to identify a new pattern in a URL with a machine learning algorithm (Text mining)

I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure. domain = ...
1
vote
0answers
46 views

Bayesian learning of tree distribution

this is my first post here. I'm currently trying to compute the posterior predictive likelihood for a tree-structured distribution, following the paper Tractable Bayesian learning of tree belief ...
0
votes
2answers
81 views

Estimating probabilities using Bayes rule?

I am working on a past exam paper. I am given a data set as follows: Hair {brown, red} = {B,R}, Height {tall, short} = {T,S} and Country {UK, Italy} = {U,I} (B,T,U) (B,T,U) (B,T,I) (R,T,U) (R,T,U) ...
3
votes
0answers
111 views

Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
1
vote
1answer
149 views

what should be the parametric form of the l2 regularization in a Bayesian setting?

In a Bayesian setting for parameter estimation, what should be the parametric form of the prior distribution in order to perform l2 regularization?
1
vote
0answers
608 views

Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
2
votes
1answer
140 views

Learn a joint distribution from incomplete samples

Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are ...
3
votes
2answers
222 views

Properties of conditional probability distributions

This is a problem from a machine learning pset that I'm self-learning from http://www.seas.harvard.edu/courses/cs281/assignment-1.pdf. Suppose we are provided with a hierarchy of three ...
0
votes
2answers
252 views

Can anyone tell me why we always use the Gaussian distribution in Machine learning?

For example, we always assumed that the data or signal error is a Gaussian distribution? why? I have asked this question on stackoverflow, the link: ...
0
votes
0answers
104 views

Adding training examples to Bayesian classifier reduces accuracy

I'm working on a problem to predict/classify overall sentiment of a large amount of text, which I can verify on the next day. Each data point is a day and is composed of multiple articles. I bin the ...