3
votes
0answers
26 views

What is the posterior probability of the data given the model used for model averaging with Bayesian Logistic Regression?

I am trying to learn about Bayesian Model Averaging using Bayesian Logistic Regression (Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text ...
4
votes
1answer
132 views

Bayesian MLPs using the MCMC methods - any tricks of the trade?

Having used the NETLAB library for MATLAB to implement Bayesian Multi-Layer Perceptron (MLP) neural networks using MacKay's evidence framework, I am now experimenting with Markov Chain Monte Carlo ...
3
votes
0answers
46 views

Reconstruct a “blocky” picture?

Consider a finite set $A$. Let the sample space be $A\times A$. We have an unknown probability distribution $f$ on this sample space. Now this probability distribution has a "blocky" property, which I ...
0
votes
1answer
59 views

Sufficient number of sample to learn Bayesian network?

I want to construct Bayesian network for a 800 genes(genes are my node/variables). I have only 30 cancer samples and 30 normal sample.so I want to create network for cancer samples and for the normal ...
1
vote
1answer
81 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
3
votes
2answers
142 views

What is the relationship between graphical models and hierarchical Bayesian models?

I've searched a good bunch of literature but have failed to find an exact distinction between the two. My impression is that in the Machine Learning literature you'll find allusions to hierarchical ...
0
votes
2answers
97 views

Direct Sampling from posterior distribution

Why is direct sampling from the posterior distribution intractable?
0
votes
0answers
44 views

Intuitive understanding of Local Probability Distribution

I'm learning Bayesian network. I have problem in intuitive understanding of Local Probability Distribution. Can anybody explain to me what it is?
1
vote
0answers
55 views

Variational Posterior Dirichlets in LDA

I am running the c code for LDA provided on David Blei's website. The code outputs several files. The output file final.gamma is supposed to include the "Variational Posterior Dirichlets". If I ...
2
votes
0answers
102 views

Good libraries for working with probabilistic graphical models?

Could someone recommend some well-maintained and up-to-date libraries for working with probabilistic graphical models? I noticed that there are some libraries for R listed here and one for C++, but ...
8
votes
1answer
123 views

regarding conditional independence and its graphical representation

When studying covariance selection, I once read the following example. With respect to the following model: Its covariance and inverse covariance matrix are given as follows, I do not understand ...
1
vote
0answers
239 views

How to compute the maximum a posteriori probability (MAP) estimate with / without a prior

I am a newbie in this area so I hope someone could explain the following problem to me in plain English. Assume I want to use MAP to estimate some parameters on the basis of some observations. I know ...
0
votes
1answer
165 views

Estimating parameters in multivariate classification resulting zero determinant sample covariance matrix

Newbie here typesetting my question, so excuse me if this don't work. I am trying to give a bayesian classifier for a multivariate classification problem where input is assumed to have multivariate ...
1
vote
1answer
132 views

Are posterior probabilities from a Naive Bayes classifier reliable?

I have read that the posterior probabilities of Naive Bayes classifiers are unreliable. Is this true? and if so, in what sense, and why? Specifically, I am interested to know if the probabilities can ...
4
votes
0answers
100 views

What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
0
votes
0answers
50 views

ML: A generalized human voting/recommendation system

I'd like to create a certain kind of voting/recommendation system. I'm sure there must be a name for what I'm trying to do, but I'm not sure. Basically, I start with a distribution of binary vectors ...
2
votes
1answer
75 views

The meaning of convergence in Variational Inference?

My friend and I are discussing about the convergence of Variational Inference, especial for Expectation Propagation method. After running some loops, the likelihood of my graphical model can be ...
1
vote
1answer
114 views

Why doesn't ML point estimate equal MAP point estimate even though I'm using uniform prior?

I asked a previous question about why the ML and MAP estimates are the same when using a uniform prior (How does a uniform prior lead to the same estimates from maximum likelihood and mode of ...
1
vote
1answer
289 views

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
0
votes
1answer
75 views

Trouble reading multinomial naive bayes notation

$C_{MAP}$: most likely class (i.e., "maximum a posteriori") $C_{NB}$: Naive Bayes x: document d is represented as $x_n$ ...
1
vote
0answers
46 views

How do I incorporate personalization to a Bayesian ranking engine?

I'm looking to quickly get smart on how to add personalization into a Bayesian-based recommendation system. I'm using clickstream data and Bayesian statistics to estimate probabilities of purchase ...
3
votes
1answer
277 views

Neural Network Black Box Workarounds

I am dealing with a data set that includes rich textual data (e.g., blog entries, magazine articles, essays, book reviews, etc.) as well as a host of proprietary metrics, including numerous ...
8
votes
2answers
1k views

Next steps after “Bayesian Reasoning and Machine Learning”

I'm currently going through "Bayesian Reasoning and Machine Learning" by David Barber and it is an extremely well written and engaging book for learning the fundamentals. So a question to someone who ...
0
votes
0answers
79 views

Predicting with Relevance Vector Machines

I am trying out this Matlab toolbox for Relevance Vector Machines by Tipping: http://www.miketipping.com/sparsebayes.htm This has an implementation of Relevance Vector Machines, and generates pretty ...
2
votes
0answers
49 views

Learning parameters of non-parametric Bayesian models

I have a sample of Chinese restaurant process which I want to model as Pitman–Yor process. How do I determine parameters of Pitman-Yor model from given sample? For Dirichlet process I would just use ...
1
vote
3answers
98 views

How can I use Bayes rule for this question given additional data

I am required to use the Naive Bayes classifier to classify example 8, to see whether it is poisonous or not. I gained the following results: p(x|Poisonous=Y) = 0.0267857 and p(x|Poisonous=N) = ...
0
votes
0answers
98 views

Understanding the derivation of an equation in LDA modeling

When reading the derivation of LDA models, I usually get the following equations. I do not quite understand the second step, where $p(\mathbf{z}_{-i},\mathbf{w}|\alpha,\beta)$ was removed. Is that ...
0
votes
0answers
88 views

How to identify a new pattern in a URL with a machine learning algorithm (Text mining)

I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure. domain = ...
1
vote
0answers
45 views

Bayesian learning of tree distribution

this is my first post here. I'm currently trying to compute the posterior predictive likelihood for a tree-structured distribution, following the paper Tractable Bayesian learning of tree belief ...
0
votes
2answers
80 views

Estimating probabilities using Bayes rule?

I am working on a past exam paper. I am given a data set as follows: Hair {brown, red} = {B,R}, Height {tall, short} = {T,S} and Country {UK, Italy} = {U,I} (B,T,U) (B,T,U) (B,T,I) (R,T,U) (R,T,U) ...
3
votes
0answers
107 views

Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
1
vote
1answer
140 views

what should be the parametric form of the l2 regularization in a Bayesian setting?

In a Bayesian setting for parameter estimation, what should be the parametric form of the prior distribution in order to perform l2 regularization?
1
vote
0answers
511 views

Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
2
votes
1answer
120 views

Learn a joint distribution from incomplete samples

Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are ...
3
votes
2answers
201 views

Properties of conditional probability distributions

This is a problem from a machine learning pset that I'm self-learning from http://www.seas.harvard.edu/courses/cs281/assignment-1.pdf. Suppose we are provided with a hierarchy of three ...
0
votes
2answers
225 views

Can anyone tell me why we always use the Gaussian distribution in Machine learning?

For example, we always assumed that the data or signal error is a Gaussian distribution? why? I have asked this question on stackoverflow, the link: ...
0
votes
0answers
101 views

Adding training examples to Bayesian classifier reduces accuracy

I'm working on a problem to predict/classify overall sentiment of a large amount of text, which I can verify on the next day. Each data point is a day and is composed of multiple articles. I bin the ...
3
votes
2answers
378 views

Bayes decision boundary of Figure 2.5 in Elements of Statistical Learning

When I read "Elements of Statistical Learning", I met some difficulty in calculating the Bayes decision boundary of Figure 2.5. In the package ElemStatLearn, it ...
2
votes
2answers
198 views

What is the difference between Informative (IVM) and Relevance (RVM) vector machines

I'm trying to understand if there is any specific difference between Informative IVMs and Relevance RVMs other than the terminology. I've not seen anything explicit. When I'm reading about vector ...
4
votes
1answer
1k views

How to write a poker player using Bayes networks

This is my first question on stackexchange and also my first time implementing a Bayesian network so I will apologize ahead of time for any novice mistakes I make. The goal of my project is to ...
2
votes
2answers
218 views

What is the appropriate machine learning algorithm for this problem?

I am working on a problem which looks like this: Input Variables Categorical a b c d Continuous e Output Variables Discrete(Integers) v x y Continuous z The major issue that I am ...
3
votes
2answers
130 views

Validation techniques for hierarchical model

I have a hierarchical model that I need to validate. My model is as follows: we have a collection of $\lambda_i$ that we draw from $Gamma(\alpha,\beta)$. Then, we draw our data point $y_i$ from ...
1
vote
0answers
36 views

Are there any ways to update SVM model incrementally like Bayesian or k-NN classifiers? [duplicate]

Possible Duplicate: Can SVM do stream learning one example at a time? It takes 30 minutes to create SVM model from the whole dataset. The training time is growing as I get more new samples. ...
5
votes
1answer
2k views

The input parameters for using latent Dirichlet allocation

When using topic modeling (Latent Dirichlet Allocation), the number of topics is an input parameter that the user need to specify. Looks to me that we should also provide a collection of candidate ...
10
votes
2answers
4k views

Two R packages for topic modeling, LDA and topicmodels?

It seems that there have two R packages for running Latent Dirichlet Allocation. One is LDA, authored by Jonathan Chang; and another is authored by Bettina Grün and Kurt Hornik. What are the ...
0
votes
1answer
142 views

Measuring information content of a random variable in Naive Bayes classifier

I'm trying to improve accuracy in a Naive Bayes classifier that uses a bunch of features. I have a hunch that removing some features may actually improve performance. My reasoning is for a ...
6
votes
2answers
335 views

How are classifications merged in an ensemble classifier?

How does an ensemble classifier merge the predictions of its constituent classifiers? I'm having difficulty finding a clear description. In some code examples I've found, the ensemble just averages ...
6
votes
3answers
1k views

Are there any tutorials on Bayesian probability theory or graphical models by example?

I've seen references to learning Bayesian probability theory in R, and I was wondering if there is more like this, perhaps specifically in Python? Geared towards learning Bayesian probability theory, ...
1
vote
1answer
487 views

Decision boundaries and Gaussian density functions

This is for my hw, and if anyone can solve the first part of the question it will be great. Here is the question: Assume a two-class problem with equal a priori class probabilities and Gaussian ...
3
votes
2answers
134 views

What techniques are used for empirical, stochastic simulation of a time series?

Suppose you have recorded a set of paths in the $y,t$ plane, with $y = f(t)$, $f$ is a stochastic function (i.e. there is a noise term), and $t$ might be time or some other monotonic increasing ...