Tagged Questions

1
vote
1answer
71 views

Next steps after “Bayesian Reasoning and Machine Learning”

I'm currently going through "Bayesian Reasoning and Machine Learning" by David Barber and it is an extremely well written and engaging book for learning the fundamentals. So a question to someone who ...
0
votes
0answers
29 views

Predicting with Relevance Vector Machines

I am trying out this Matlab toolbox for Relevance Vector Machines by Tipping: http://www.miketipping.com/sparsebayes.htm This has an implementation of Relevance Vector Machines, and generates pretty ...
1
vote
0answers
27 views

Learning parameters of non-parametric Bayesian models

I have a sample of Chinese restaurant process which I want to model as Pitman–Yor process. How do I determine parameters of Pitman-Yor model from given sample? For Dirichlet process I would just use ...
1
vote
2answers
48 views

How can I use Bayes rule for this question given additional data

I am required to use the Naive Bayes classifier to classify example 8, to see whether it is poisonous or not. I gained the following results: p(x|Poisonous=Y) = 0.0267857 and p(x|Poisonous=N) = ...
0
votes
0answers
27 views

Understanding the derivation of an equation in LDA modeling

When reading the derivation of LDA models, I usually get the following equations. I do not quite understand the second step, where $p(\mathbf{z}_{-i},\mathbf{w}|\alpha,\beta)$ was removed. Is that ...
0
votes
0answers
54 views

How to identify a new pattern in a URL with a machine learning algorithm (Text mining)

I am trying to identify new patterns after analyzing a number of URLs. So let's say, I am investigating the hypothetical website Yoohle.com and their URLs have the following structure. domain = ...
1
vote
0answers
34 views

Bayesian learning of tree distribution

this is my first post here. I'm currently trying to compute the posterior predictive likelihood for a tree-structured distribution, following the paper Tractable Bayesian learning of tree belief ...
0
votes
2answers
75 views

Estimating probabilities using Bayes rule?

I am working on a past exam paper. I am given a data set as follows: Hair {brown, red} = {B,R}, Height {tall, short} = {T,S} and Country {UK, Italy} = {U,I} (B,T,U) (B,T,U) (B,T,I) (R,T,U) (R,T,U) ...
3
votes
0answers
85 views

Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
1
vote
1answer
115 views

what should be the parametric form of the l2 regularization in a Bayesian setting?

In a Bayesian setting for parameter estimation, what should be the parametric form of the prior distribution in order to perform l2 regularization?
0
votes
0answers
125 views

Unscented Kalman Filter-Negative Covariance Matirx

I have recently started working on the Unscented Kalman Filter. I coded the numerically stable version (i.e Square root Kalman filter) and use matlab for implementing. In the final update step , ...
2
votes
0answers
57 views

Learn a joint distribution from incomplete samples

Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are ...
3
votes
2answers
114 views

Properties of conditional probability distributions

This is a problem from a machine learning pset that I'm self-learning from http://www.seas.harvard.edu/courses/cs281/assignment-1.pdf. Suppose we are provided with a hierarchy of three ...
0
votes
2answers
170 views

Can anyone tell me why we always use the Gaussian distribution in Machine learning?

For example, we always assumed that the data or signal error is a Gaussian distribution? why? I have asked this question on stackoverflow, the link: ...
0
votes
0answers
73 views

Adding training examples to Bayesian classifier reduces accuracy

I'm working on a problem to predict/classify overall sentiment of a large amount of text, which I can verify on the next day. Each data point is a day and is composed of multiple articles. I bin the ...
3
votes
1answer
287 views

Bayes decision boundary of Figure 2.5 in Elements of Statistical Learning

When I read "Elements of Statistical Learning", I met some difficulty in calculating the Bayes decision boundary of Figure 2.5. In the package ElemStatLearn, it ...
2
votes
2answers
155 views

What is the difference between Informative (IVM) and Relevance (RVM) vector machines

I'm trying to understand if there is any specific difference between Informative IVMs and Relevance RVMs other than the terminology. I've not seen anything explicit. When I'm reading about vector ...
4
votes
1answer
554 views

How to write a poker player using Bayes networks

This is my first question on stackexchange and also my first time implementing a Bayesian network so I will apologize ahead of time for any novice mistakes I make. The goal of my project is to ...
2
votes
2answers
189 views

What is the appropriate machine learning algorithm for this problem?

I am working on a problem which looks like this: Input Variables Categorical a b c d Continuous e Output Variables Discrete(Integers) v x y Continuous z The major issue that I am ...
3
votes
2answers
96 views

Validation techniques for hierarchical model

I have a hierarchical model that I need to validate. My model is as follows: we have a collection of $\lambda_i$ that we draw from $Gamma(\alpha,\beta)$. Then, we draw our data point $y_i$ from ...
1
vote
0answers
33 views

Are there any ways to update SVM model incrementally like Bayesian or k-NN classifiers? [duplicate]

Possible Duplicate: Can SVM do stream learning one example at a time? It takes 30 minutes to create SVM model from the whole dataset. The training time is growing as I get more new samples. ...
3
votes
1answer
473 views

The input parameters for using latent Dirichlet allocation

When using topic modeling (Latent Dirichlet Allocation), the number of topics is an input parameter that the user need to specify. Looks to me that we should also provide a collection of candidate ...
8
votes
2answers
2k views

Two R packages for topic modeling, LDA and topicmodels?

It seems that there have two R packages for running Latent Dirichlet Allocation. One is LDA, authored by Jonathan Chang; and another is authored by Bettina Grün and Kurt Hornik. What are the ...
0
votes
1answer
121 views

Measuring information content of a random variable in Naive Bayes classifier

I'm trying to improve accuracy in a Naive Bayes classifier that uses a bunch of features. I have a hunch that removing some features may actually improve performance. My reasoning is for a ...
4
votes
2answers
237 views

How are classifications merged in an ensemble classifier?

How does an ensemble classifier merge the predictions of its constituent classifiers? I'm having difficulty finding a clear description. In some code examples I've found, the ensemble just averages ...
5
votes
2answers
812 views

Are there any tutorials on bayesian probability theory or graphical models by example?

I've seen references to learning Bayesian probability theory in R, and I was wondering if there is more like this, perhaps specifically in Python? Geared towards learning Bayesian probability theory, ...
1
vote
1answer
317 views

Decision boundaries and Gaussian density functions

This is for my hw, and if anyone can solve the first part of the question it will be great. Here is the question: Assume a two-class problem with equal a priori class probabilities and Gaussian ...
3
votes
2answers
117 views

What techniques are used for empirical, stochastic simulation of a time series?

Suppose you have recorded a set of paths in the $y,t$ plane, with $y = f(t)$, $f$ is a stochastic function (i.e. there is a noise term), and $t$ might be time or some other monotonic increasing ...
2
votes
0answers
71 views

Problem on parametric learning when datasets are small

I'm currently writing a program to learn a TAN (Tree-Augmented Bayesian network) classifier from data, and I have almost finished it. I use the algorithm described in Friedman's paper 'Bayesian ...
14
votes
4answers
267 views

Addressing model uncertainty

I was wondering how the Bayesians in the CrossValidated community view the problem of model uncertainty and how they prefer to deal it? I will try to pose my question in two parts: 1) How important ...
0
votes
2answers
91 views

Viewing kernel regression in a Bayesian framework

If one wanted to use Kernel Regression in a Bayesian Framework, any ideas on how one would go about it? Kernel Regression
5
votes
2answers
316 views

Deriving mathematical model of pLSA

After knowing how LSA works, I went on continue reading on pLSA but couldn't really make sense of the mathematical formula. This is what I get from wikipedia (other academic papers/tutorial show ...
1
vote
0answers
180 views

Confusion in MLE and EM [closed]

I was trying to read through Maximum Likelihood Estimation(MLE) and Expectation and Maximization(EM) algorithm. But while reading them, I got two interpretations. I am trying to post my questions, ...
15
votes
2answers
504 views

How to choose between learning algorithms

I need to implement a program that will classify records into 2 categories (true/false) based on some training data, and I was wondering at which algorithm/methodology I should be looking at. There ...
2
votes
2answers
211 views

Resources about Gibbs sampling in hybrid Bayesian networks

Greetings, I've been trying to get my hands on a substantial resource for using Gibbs sampling in hybrid Bayesian networks, that is, networks with both continuous and discrete variables. So far I ...
0
votes
0answers
326 views

Question On Naive Bayes, SVM, HMM and CRF [closed]

Dear Group, I am trying to implement one shallow parser and one deep parser and trying to implement them in the four models namely, Naive Bayes, SVM, HMM and CRF. I am trying to implement in each one ...
5
votes
1answer
443 views

How to compute prediction error from Relevance Vector Machine and Gaussian Process Regression?

Does anyone know how to construct a confidence interval for predicting a new test value given a trained Relevance Vector Machine (rvm) and/or Gaussian Process ...
6
votes
1answer
89 views

Correlation between two nodes of a single layer MLP for joint-Gaussian input

Let's say you have a jointly gaussian vector random variable $\mathbf{x}$, with mean $\mathbf{M}$ and covariance $\mathbf{S}$. I now transform each scalar element of $\mathbf{x}$ , say $x_j$, with a ...
10
votes
12answers
8k views

Best books for an introduction to statistical data analysis?

I bought this book: How to Measure Anything: Finding the Value of Intangibles in Business and Head First Data Analysis: A Learner's Guide to Big Numbers, Statistics, and Good Decisions What ...