Questions tagged [nonparametric-bayes]

Bayesian methods for infinite dimensional parameter spaces.

Filter by
Sorted by
Tagged with
0
votes
1answer
406 views

Likely mean of a multinomial distribution with dirichlet prior

I am working to create a Bayesian non-parametric estimate of the mean of a distribution given a distribution of observations. Ultimately I'd like to get to a credibility interval of the likely mean of ...
1
vote
1answer
279 views

For inference of Dirichlet Process Mixture, why the expected value $\int h(x)f(x)$ is desired?

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture? $f(x)$ is the probability density function, ...
1
vote
0answers
37 views

Is it a problem to have non homogeneus sampling time in Bayes Filter?

I have a doubt related with Recursive State Estimation using Bayes Filter (actually using an aproximation to that through Particle Filters) This algorithm is explained in several sources with ...
25
votes
3answers
3k views

Is there a Bayesian approach to density estimation

I am interested to estimate the density of a continuous random variable $X$. One way of doing this that I learnt is the use of Kernel Density Estimation. But now I am interested in a Bayesian ...
0
votes
1answer
1k views

Understanding the difference between Supervised and unsupervised learning?

I have been reading about the Supervised and Unsupervised learning. What I came to know through this link is that in case of Supervised learning you have a set of input and a set of labels which are ...
7
votes
3answers
1k views

Books for learning non parametric Bayesian model

Having studied parametric Bayesian statistics during the two last years, I plan to begin to self-study non parametric Bayesian model during this summer and look for recommendations. I would like the ...
5
votes
1answer
2k views

understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for ...
4
votes
1answer
4k views

stick breaking model of Dirichlet process

I have a question regarding sticking-breaking model of Dirichlet process, which is defined as follows: There are further statements that I am not clear that how to derive equation 1 from that ...
2
votes
0answers
32 views

Is there Bayesian search theory about search through the space of Bayesian models?

Consider we start with a specific Bayesian model, say an infinite mixture model with a Dirichlet Process as a prior. I know there are wildly many variants on this theme, from the Hierarchical ...
1
vote
0answers
58 views

How well can the Dirichlet process cluster really small datasets?

I have been debating between a model-based parametric clustering approach (e.g. HMMs), and a hierarchical Dirichlet/Pitman-Yor process for clustering sequential data. I understand the latter has been ...
1
vote
1answer
395 views

How to draw samples from a Bayesian nonparamatric density estimation? [DPpackage]

I am trying to compute a Kernel Density from high dimensional data ($n > 2$). The underlying (generative) model is assumed unknown. The goal is to draw samples from this estimate, in a sense ...
1
vote
1answer
410 views

Why semi/nonparametric models?

Increasing the flexibility of models makes it prone to overfitting. On the other hand, it looks to me that, if the space function classes $\mathcal{F}$ is too big, it is hard to prove bounds on ...
10
votes
2answers
6k views

PyMC for nonparametric clustering: Dirichlet process to estimate Gaussian mixture's parameters fails to cluster

Problem setup One of the first toy problems I wanted to apply PyMC to is nonparametric clustering: given some data, model it as a Gaussian mixture, and learn the number of clusters and each cluster's ...
1
vote
0answers
106 views

How to sample via blocked Gibbs the conjugate normal model?

i am trying to sample the two dimensional parameter $\theta=(\mu,\sigma^2)$, for the Normal model. I have the full conditionals being for the mean: $\pi(\mu_{j}|\ldots) \sim N\left(\frac{\xi_{j}\...
15
votes
4answers
10k views

Gaussian Processes: How to use GPML for multi-dimensional output

Is there a way to perform Gaussian Process Regression on multidimensional output (possibly correlated) using GPML? In the demo script I could only find a 1D example. A similar question on CV that ...
1
vote
0answers
182 views

In GPML, why does scaler in Squared Exponent covariance function gets very small?

I'm running regression using GPML with a covariance function being a sum of a Gaussian noise and a Squared Exponent (SE). Input is in R4 and both the input and the output are normalized. I run ...
3
votes
0answers
233 views

Estimating parameters of categorical distribution from sum-of-outcome data

Let $X$ be a categorical random variable with possible outcomes $o_1,...,o_n \subset [l, u]$ (real numbers with a known lower bound $l>0$ and known upper bound $u$) that occur with probability $p(X ...
3
votes
0answers
1k views

Non-parametric estimate of conditional expectation

I have a (fairly smooth) function $f$ and a sample $\{(x_i,y_i)\}_{i=1,\ldots,N}$ from the joint distribution of the random variables $X$ and $Y$. I would like to estimate the conditional expectation ...
2
votes
0answers
158 views

Points to keep in mind while implementing a nonparametric bayesian inference procedure from scratch

I have been trying to implement a Bayesian inference procedure from scratch for a specific problem, but I have implemented the procedure, and it doesn't seem to work. Since, I can't just post the ...
10
votes
1answer
2k views

Putting a prior on the concentration parameter in a Dirichlet process

Most of this is background, skip to the end if you already know enough about Dirichlet process mixtures. Suppose I am modeling some data as coming from a mixture of Dirichlet processes, i.e. let $F \...
10
votes
2answers
2k views

Introductory textbook on nonparametric Bayesian models?

I'd like to wrap my head around this topic but learning from white-papers and tutorials is hard because there are many gaps which are usually filled in textbooks. If it is important I have ...
4
votes
1answer
125 views

Learning parameters of non-parametric Bayesian models

I have a sample of Chinese restaurant process which I want to model as Pitman–Yor process. How do I determine parameters of Pitman-Yor model from given sample? For Dirichlet process I would just use ...
0
votes
1answer
960 views

NMDS for biomass

I would like to make a NMDS with biomass of different prey groups in stomach content of fish. I have already made one where the data matrix consists of 0 and 1, and this one went fine but are not ...
6
votes
2answers
638 views

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
11
votes
1answer
3k views

Covariance matrix for Gaussian Process and Wishart distribution

I'm reading through this paper on Generalised Wishart Processes (GWP). The paper calculates the covariances between different random variables (following Gaussian Process) using squared exponential ...
1
vote
0answers
73 views

Nonparametric Bayesian priors on mean $0$ distributions?

Is there any standard way of putting a prior on the mean $0$ distributions? I'm interested in this from the perspective of robustly modelling the error distribution in a regression. So for instance I ...
4
votes
1answer
898 views

Bayesian Updating for a Discrete Rating Value

I have an item for which I slowly collect rating values on a website. It is a movie item on a website and at the beginning it has no rating but I assign it a Gaussian prior $N(\mu_0, \sigma_0^2)$. A ...
0
votes
1answer
5k views

Are Bayesian approches used for classification (supervised) or for clustering (unsupervised)?

Are Bayesian approaches (static and dynamic) used for classification (which is supervised) or for clustering (which is unsupervised)? or can they be used for both ? I even see that for instance to ...
3
votes
0answers
235 views

Default priors for mixtures

Suppose we have $p$ dimensional vectors $Y_i$ which we model with $f_Y (y |\theta) = \sum \pi_k N(y | \mu_k, \Sigma_k)$ with $\theta$ being a catch all for the model parameters (the number of ...
7
votes
2answers
2k views

How adding covariance noise in Gaussian processes to prevent overfitting?

I am told, in Gaussian Processes, adding covariance function noise to others, say SEiso or Materns, cause a better result, since it prevents from over fitting. I appreciate if someone could put more ...
4
votes
1answer
1k views

Pitman-Yor processes in R or Python

I am looking for a good tool in R or Python or any other implementation that can help to me generate sampling from hierarchical Pitman-Yor processes (HPY) (one of the recent and popular nonparametric ...
5
votes
5answers
5k views

Simple introduction to MCMC with Dirichlet process prior?

I'm looking for a simple and easy to read introduction to using MCMC with a dirichlet process prior. Or perhaps using MCMC in any machine learning scenario, eg Gaussian Process. I've been circling ...
1
vote
2answers
302 views

How do I compare multiple arbitrary predictions for a given data set?

Background: I am developing a Python Statistics Framework, not because the ones out there are bad but because it will help me learn Python and Statistics. I have taken AP Stats, and read scattered ...
6
votes
2answers
1k views

What are some applications of Chinese restaurant processes?

What are some applications of Chinese restaurant processes? I'm trying to learn a bit about non-parametric Bayesian methods, starting with Dirichlet processes and CRPs, but all the tutorials I've ...
5
votes
1answer
5k views

How do I use the GPML package for multi dimensional input?

I have downloaded the Gaussian Processes for Machine Learning (GPML) package (gpml-matlab-v3.1-2010-09-27.zip) from the website, and I can run the regression example (demoRegression) in Octave. It ...

1
2