# Tagged Questions

35 views

### SVM versus Bayesian regression example(s)?

I am trying to track down examples where some basic problems have been tackled via both classical Machine Learning algorithms and more formal statistical methods. In particular, I'm interested in ...
54 views

### Mixture of probits: understanding truncated-based likelihoods

I am trying to implement a mixture model of probits to infer the best decision boundary for every latent subpopulation. When doing Gibbs sampling, we eventually have to compute $P(y^* | w_c)$ where ...
25 views

### Bayesian Perceptron - how can I generate many different perceptrons?

I am going to implement the Bayesian version of a perceptron that I read in the Statistical Mechanics of learning, by Engel-Van Den Broeck. The idea to improve the performance is to use many Gibbs ...
180 views

### Stochastic Programming with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
169 views

### What is the Probability Distribution of NLTK Naive Bayes?

As I know Naïve Bayes has various distributions, as said in Sci-kit learn manual: The different naive Bayes classifiers differ mainly by the assumptions they make regarding the distribution of ...
47 views

### What is the difference between Binary Clasification and Multiclass classification?

Apology for posting almost one question daily. I am trying to learn some aspects of Statistical Machine learning, so every day many questions coming and if I am not finding answer in my offline peer ...
33 views

### What machine learning tool is best suited for taking time series data as well as descriptive data and making a binomial classification

I have an interesting task of utilizing log data from computer servers in a server farm and predicting if a particular server is likely to fail in the next 24 hours. My data set will be comprised of ...
47 views

### Estimation in Naive Bayes

I have a very silly question. In Multinomial Naive Bayes Classifier, which parameter estimation do we use, is it Maximum Likelihood or Maximum A Posteriori? If any one of the esteemed members may ...
33 views

### What does it take to learn a Model?

In Machine Learning, there are various models. I tried to learn few probabilistic models of Machine Learning. I read the theory, worked on problems and tested my results and could analyze my data ...
62 views

### How to construct a reasonable prior and likelihood for Bayes modelling?

To apply Bayes inference for data analysis or machine learning, we have to construct prior and likelihood, right? But if I fail to come up with a reasonable prior and likelihood, then the Bayes model ...
124 views

### Intuitive explanation of Bayesian logistic regression?

I'm looking for an intuitive explanation of Bayesian Logistic Regression (I'm using it for texts if that's relevant). It seems that this article presents it, but it's, uh, way too mathy. Thanks!
46 views

### What are the general strategies in creating a Probabilistic Graphical Model?

While there is lot of theory and probability in the background to understand, I wanted to know if there are any resources/quick pointers on what to consider while modeling a problem using Bayesian ...
65 views

### difference between hierarchical dirichlet process and nested dirichlet process

There have some extensions to Dirichlet process. One is Hierarchical Dirichlet process, and another is Nested Dirichlet Process. What are the differences between these two? I once read the paper of ...
130 views

### ML vs MAP estimation, when to use which?

ML = Maximum Liklihood MAP = Maximum a-posteriori ML is intuitive/naive in that it starts only with the probability of observation given the parameter (i.e. the likelihood function) and tries to ...
99 views

### understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for ...
44 views

### What is the posterior probability of the data given the model used for model averaging with Bayesian Logistic Regression?

I am trying to learn about Bayesian Model Averaging using Bayesian Logistic Regression (Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text ...
203 views

### Bayesian MLPs using the MCMC methods - any tricks of the trade?

Having used the NETLAB library for MATLAB to implement Bayesian Multi-Layer Perceptron (MLP) neural networks using MacKay's evidence framework, I am now experimenting with Markov Chain Monte Carlo ...
47 views

### Reconstruct a “blocky” picture?

Consider a finite set $A$. Let the sample space be $A\times A$. We have an unknown probability distribution $f$ on this sample space. Now this probability distribution has a "blocky" property, which I ...
71 views

### Sufficient number of sample to learn Bayesian network?

I want to construct Bayesian network for a 800 genes(genes are my node/variables). I have only 30 cancer samples and 30 normal sample.so I want to create network for cancer samples and for the normal ...
143 views

### Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
188 views

### What is the relationship between graphical models and hierarchical Bayesian models?

I've searched a good bunch of literature but have failed to find an exact distinction between the two. My impression is that in the Machine Learning literature you'll find allusions to hierarchical ...
112 views

### Direct Sampling from posterior distribution

Why is direct sampling from the posterior distribution intractable?
47 views

### Intuitive understanding of Local Probability Distribution

I'm learning Bayesian network. I have problem in intuitive understanding of Local Probability Distribution. Can anybody explain to me what it is?
72 views

### Variational Posterior Dirichlets in LDA

I am running the c code for LDA provided on David Blei's website. The code outputs several files. The output file final.gamma is supposed to include the "Variational Posterior Dirichlets". If I ...
213 views

### Good libraries for working with probabilistic graphical models?

Could someone recommend some well-maintained and up-to-date libraries for working with probabilistic graphical models? I noticed that there are some libraries for R listed here and one for C++, but ...
132 views

### regarding conditional independence and its graphical representation

When studying covariance selection, I once read the following example. With respect to the following model: Its covariance and inverse covariance matrix are given as follows, I do not understand ...
403 views

### How to compute the maximum a posteriori probability (MAP) estimate with / without a prior

I am a newbie in this area so I hope someone could explain the following problem to me in plain English. Assume I want to use MAP to estimate some parameters on the basis of some observations. I know ...
195 views

### Estimating parameters in multivariate classification resulting zero determinant sample covariance matrix

Newbie here typesetting my question, so excuse me if this don't work. I am trying to give a bayesian classifier for a multivariate classification problem where input is assumed to have multivariate ...
235 views

### Are posterior probabilities from a Naive Bayes classifier reliable?

I have read that the posterior probabilities of Naive Bayes classifiers are unreliable. Is this true? and if so, in what sense, and why? Specifically, I am interested to know if the probabilities can ...
111 views

### What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
50 views

### ML: A generalized human voting/recommendation system

I'd like to create a certain kind of voting/recommendation system. I'm sure there must be a name for what I'm trying to do, but I'm not sure. Basically, I start with a distribution of binary vectors ...
97 views

### The meaning of convergence in Variational Inference?

My friend and I are discussing about the convergence of Variational Inference, especial for Expectation Propagation method. After running some loops, the likelihood of my graphical model can be ...
128 views

### Why doesn't ML point estimate equal MAP point estimate even though I'm using uniform prior?

I asked a previous question about why the ML and MAP estimates are the same when using a uniform prior (How does a uniform prior lead to the same estimates from maximum likelihood and mode of ...
379 views

### How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
81 views

### Trouble reading multinomial naive bayes notation

$C_{MAP}$: most likely class (i.e., "maximum a posteriori") $C_{NB}$: Naive Bayes x: document d is represented as $x_n$ ...
52 views

### How do I incorporate personalization to a Bayesian ranking engine?

I'm looking to quickly get smart on how to add personalization into a Bayesian-based recommendation system. I'm using clickstream data and Bayesian statistics to estimate probabilities of purchase ...
281 views

### Neural Network Black Box Workarounds

I am dealing with a data set that includes rich textual data (e.g., blog entries, magazine articles, essays, book reviews, etc.) as well as a host of proprietary metrics, including numerous ...
2k views

### Next steps after “Bayesian Reasoning and Machine Learning”

I'm currently going through "Bayesian Reasoning and Machine Learning" by David Barber and it is an extremely well written and engaging book for learning the fundamentals. So a question to someone who ...
72 views

### Learning parameters of non-parametric Bayesian models

I have a sample of Chinese restaurant process which I want to model as Pitman–Yor process. How do I determine parameters of Pitman-Yor model from given sample? For Dirichlet process I would just use ...
109 views

### How can I use Bayes rule for this question given additional data

I am required to use the Naive Bayes classifier to classify example 8, to see whether it is poisonous or not. I gained the following results: p(x|Poisonous=Y) = 0.0267857 and p(x|Poisonous=N) = ...
143 views

### Understanding the derivation of an equation in LDA modeling

When reading the derivation of LDA models, I usually get the following equations. I do not quite understand the second step, where $p(\mathbf{z}_{-i},\mathbf{w}|\alpha,\beta)$ was removed. Is that ...
93 views

### How to identify a new pattern in a URL with a machine learning algorithm (Text mining)

I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure. domain = ...
47 views

### Bayesian learning of tree distribution

this is my first post here. I'm currently trying to compute the posterior predictive likelihood for a tree-structured distribution, following the paper Tractable Bayesian learning of tree belief ...
82 views

### Estimating probabilities using Bayes rule?

I am working on a past exam paper. I am given a data set as follows: Hair {brown, red} = {B,R}, Height {tall, short} = {T,S} and Country {UK, Italy} = {U,I} (B,T,U) (B,T,U) (B,T,I) (R,T,U) (R,T,U) ...
115 views

### Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
152 views

### what should be the parametric form of the l2 regularization in a Bayesian setting?

In a Bayesian setting for parameter estimation, what should be the parametric form of the prior distribution in order to perform l2 regularization?
657 views

### Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
143 views

### Learn a joint distribution from incomplete samples

Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are ...
254 views

### Properties of conditional probability distributions

This is a problem from a machine learning pset that I'm self-learning from http://www.seas.harvard.edu/courses/cs281/assignment-1.pdf. Suppose we are provided with a hierarchy of three ...
271 views

### Can anyone tell me why we always use the Gaussian distribution in Machine learning?

For example, we always assumed that the data or signal error is a Gaussian distribution? why? I have asked this question on stackoverflow, the link: ...