We changed our privacy policy. Read more.

Questions tagged [nonparametric-bayes]

Bayesian methods for infinite dimensional parameter spaces.

Filter by
Sorted by
Tagged with
3
votes
1answer
196 views

Marginalizing over a Chinese Restaurant Process prior

I am reading a paper by Kemp et al. and there is a part about marginalising over a Chinese Restaurant Process and I am quite clueless about how could one marginalise over such a prior! The details of ...
2
votes
0answers
48 views

How to interpret this plot?

Please give me an explanation based on this Non-linear estimation results from the aggregate model (HOSP). This figure shows the non-linear effect of age (AGE), education (EDU), family size (FAMSZ), ...
0
votes
0answers
14 views

Are the Beta and Bernoulli Processes Normalized Random Measures with Independent Increments?

James et al. 2009 introduced the notion of "Normalized Random Measures with Independent Increments." Do the Beta process and Bernoulli process belong to this family?
1
vote
0answers
54 views

Online Stochastic Variational Inference for Dirichlet Process Mixture Models

There's a 2013 NeurIPS paper I'm trying to understand, Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation. I have a few questions: Equation 2, which defines a ...
0
votes
1answer
24 views

what does z|G mean in nonparametric bayes?

when saying z is distributed according to G, where G comes from a dirichlet process, i saw this expression: z|G ~ G is this same meaning with z ~ G ?
0
votes
1answer
37 views

Mean of draws of Dirichlet Process

Lets assume a Dirichlet process random measure in stick-breaking notation $G=\sum^\infty_{i=1} p_i \delta_{\lambda_i}$, such that $\lambda_i\sim H$ from some base distribution H, with point mass $\...
2
votes
1answer
59 views

Why nonparametric maximum likelihood of mixture is convex

Consider $x_i \sim N(\mu_i, 1)$ where $i = 1, \ldots, n$ and assume $\mu_i$ is generated i.i.d. from an unknown distribution $F$. We are interested in estimating the unknown $\mu_i$. One way to solve ...
0
votes
0answers
30 views

Reason for Poisson Distribution in Indian Buffet Process?

I have a two part question regarding the Indian Buffet Process. Suppose we consider $IBP(\alpha)$. In the IBP, the $n$th customer samples $\lambda_n \sim Poisson(\alpha/n)$ new dishes. Why does the ...
0
votes
0answers
32 views

Existence of Indian Buffet Dish Distribution?

The Chinese Restaurant Process has an associated distribution, the Chinese Restaurant Table Distribution, which describes the probability of the number of non-empty tables after N customers have been ...
2
votes
1answer
26 views

Expected Number of Dishes in Indian Buffet Process?

I'm sure this question has an answer somewhere online, but I can't find it. Suppose I have an Indian Buffet Process with $T$ customers and concentration parameter $\alpha$. For those unfamiliar with ...
3
votes
0answers
98 views

A clarification in the original Dirichlet Process paper by Ferguson

I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
4
votes
1answer
121 views

Why does Chinese Restaurant Table Distribution look like a Gaussian Distribution?

The Chinese Restaurant Table Distribution describes the probability distribution for the number of non-empty tables in the Chinese Restaurant Process after $T$ customers have been seated. Specifically,...
2
votes
1answer
114 views

Chinese Restaurant Process: Expected cardinality (number of customers) of each block (table)?

Short version of the question: The Chinese Restaurant Process defines a distribution over partitions of $[T] := \{1, ...., T\}$. What is the expected cardinality of the $t$th block, where $t \in \{1, ....
2
votes
0answers
53 views

Bayesian Model Fit for Binary Data

I am looking for methods (and preferably references) for assessing model fit for Bayesian analyses of binary data. Specifically, I am fitting Bayesian parametric and nonparametric item response models ...
1
vote
0answers
49 views

Bayesian nonparametric estimate of median [closed]

I've been working on estimating the population median of a variable with a complex distribution that is not easily characterized as a parameterized probability distribution. So far, the best I have ...
0
votes
1answer
14 views

R fitted() function applied to rbart (Bayesian Additive Regression Trees w/ random effects) object in the dbarts package

I'm having a hard time figuring out what the output of fitted() applied to a rbart object means. Specifically, I fit my data ...
1
vote
0answers
27 views

Dirichlet Process vs Hierarchical Dirichlet Process: coupling among transitions on infinite HMM

I'm new to nonparametric Bayesian, and I am reading a paper about beam sampling for the infinite hidden Markov model. In the paper, it is mentioned that since there is no coupling among the ...
0
votes
1answer
40 views

What is the Dirichlet Proces Mixture Models posterior

I am trying to understand Dirichlet Process Mixture models. One of the videos I have been watching is by Tamara Broderick. I think it is a very good introductory video to Dirichlet Process mixture ...
3
votes
1answer
47 views

Why are discrete random measures not dominated?

A statement I’ve seen without proof in many books and papers is that random probability measures obtained by normalizing a completely random measure, such as the Dirichlet process, are not dominated ...
2
votes
0answers
362 views

Bayesian Nonparametric Latent feature model

For quite a long time I've been trying to understand the paper "Bayesian Nonparametric Latent feature model" (by Zoubin Ghahramani et al.) [http://mlg.eng.cam.ac.uk/zoubin/papers/GhaGriSol06.pdf]. In ...
0
votes
0answers
40 views

Exchangabilty in CRF generative model for HDP

In the HDP Setting, the groups (or documents) are assumed exchangeable between them, and the samples (or words) within each group/topic are also exchangeable. Note the Setting chapter here However, ...
4
votes
1answer
185 views

Kernel density estimate vs Dirichlet process mixture

Nowadays the Dirichlet process mixture (DPM) seems to be the default Bayesian approach for density estimation. My question is why not simply use the kernel density estimate (KDE) to model the density? ...
4
votes
1answer
119 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
10
votes
1answer
319 views

Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?

The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a ...
10
votes
2answers
6k views

PyMC for nonparametric clustering: Dirichlet process to estimate Gaussian mixture's parameters fails to cluster

Problem setup One of the first toy problems I wanted to apply PyMC to is nonparametric clustering: given some data, model it as a Gaussian mixture, and learn the number of clusters and each cluster's ...
7
votes
2answers
868 views

Chinese Restaurant process (CRP)

I am trying to understand the Chinese Restaurant process (CRP) and Weighted Chinese Restaurant process (WCRP) described in a research paper "Automatic Discovery of Cognitive Skills"- Robert V. Lindsey,...
15
votes
4answers
10k views

Gaussian Processes: How to use GPML for multi-dimensional output

Is there a way to perform Gaussian Process Regression on multidimensional output (possibly correlated) using GPML? In the demo script I could only find a 1D example. A similar question on CV that ...
0
votes
1answer
908 views

Coding a simple Stick-Breaking Process in Python

I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python. I've got this ...
4
votes
1answer
1k views

Pitman-Yor processes in R or Python

I am looking for a good tool in R or Python or any other implementation that can help to me generate sampling from hierarchical Pitman-Yor processes (HPY) (one of the recent and popular nonparametric ...
1
vote
0answers
91 views

How to interpret graphical model for Dirichlet process mixture for variational inference?

I am working through this paper by Blei and Jordan, which introduces variational inference for Dirichlet process mixtures. They derive an evidence lower bound (ELBO) function based on a stick breaking ...
7
votes
3answers
1k views

Books for learning non parametric Bayesian model

Having studied parametric Bayesian statistics during the two last years, I plan to begin to self-study non parametric Bayesian model during this summer and look for recommendations. I would like the ...
1
vote
0answers
73 views

Nonparametric Bayesian priors on mean $0$ distributions?

Is there any standard way of putting a prior on the mean $0$ distributions? I'm interested in this from the perspective of robustly modelling the error distribution in a regression. So for instance I ...
0
votes
1answer
264 views

In a Dirichlet process, can the base distribution be discrete?

Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete. 1) If the base distribution is continuous, drawing from it will get a new value (a new ...
4
votes
1answer
1k views

Gaussian Process: vector valued response

Gaussian Processes (GPs) define a prior over functions that can be updated to a posterior once we have observed data. I've been working with scalar-valued GPs, i.e. functions $f: \mathbb{R}^{d} \...
10
votes
2answers
2k views

Introductory textbook on nonparametric Bayesian models?

I'd like to wrap my head around this topic but learning from white-papers and tutorials is hard because there are many gaps which are usually filled in textbooks. If it is important I have ...
11
votes
0answers
511 views

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth ...
0
votes
1answer
2k views

Normality test for likert scale [duplicate]

I have a questionnaire data that consist of composite Likert scale and discrete Likert items. Likert scales or variables are the sum of few Likert items while the one-item variables are represented ...
3
votes
0answers
197 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
5
votes
5answers
5k views

Simple introduction to MCMC with Dirichlet process prior?

I'm looking for a simple and easy to read introduction to using MCMC with a dirichlet process prior. Or perhaps using MCMC in any machine learning scenario, eg Gaussian Process. I've been circling ...
0
votes
0answers
35 views

DPMM asymptotics for finite mixtures?

My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
1
vote
1answer
171 views

Nonparametric topic modeling: hierarchical dirichlet vs. Indian buffet?

The hierarchical dirichlet process (Teh 2005) allows you to discover unlimited topics to describe a document. An alternative process, the Indian Buffet process (Griffiths 2011) is another ...
2
votes
0answers
434 views

Gibbs sampling in the Hierarchical Dirichlet Process

For an inference problem using a Dirichlet Process prior, one can derive a "basic" Gibbs sampling scheme, where we have a conditional for any parameter $\theta_i$ given the samples $x_i$ and all the ...
1
vote
0answers
415 views

About Hierarchical Latent Dirichlet allocation (hLDA)

I am reading Blei et al.'s paper (Hierarchical Topic Models and the Nested Chinese Restaurant Process) about hLDA. I am confused about the details of deriving the posterior of $p(\boldsymbol c_m| \...
9
votes
1answer
281 views

What does it mean to integrate over a random measure?

I'm currently looking at a paper of Dirichlet process random effects model and the model specification is as follows: $$ \begin{align*}y_{i} &= X_{i}\beta + \psi_{i} + \epsilon_{i}\\ \psi_{i} &...
2
votes
1answer
342 views

Question about implementing nested Chinese Restaurant Process (nCRP)

I am trying to follow the original paper on nCRP by Blei et al., 2010 and am confused with it's implementation. The authors describe the analogy for an nCRP as follows: A tourist arrives at the ...
4
votes
1answer
503 views

Chinese Restaurant Process

I want to implement Chinese Restaurant Process representation of Dirichlet Process for random partitions. The problem setup is as follows: I have some data (customers) which I have to randomly ...
6
votes
2answers
655 views

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be ...
2
votes
1answer
294 views

Proof of neutrality for dirichlet distribution

I am trying to learn the fields of bayesian non-parametric approaches. I am going thru this manuscript: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf I am bit stuck with: ...
3
votes
0answers
90 views

Bayesian nonparametrics vs model selection using Minimum Message Length

As we know mixture models are important tools in density estimation and in general in statistical machine learning. I have always used nonparametric Bayesian mixture models to avoid the problem of ...
0
votes
1answer
135 views

resampling hyperparameters in a Hierarchical Dirichlet Process

The sampling scheme for the hyper-parameters of hierarchical dirichlet process (HDP) is explained in the appendix of the original paper by Teh et al. I agree that the auxiliary variable $s_j$ is a ...