Questions tagged [nonparametric-bayes]
Bayesian methods for infinite dimensional parameter spaces.
85
questions
0
votes
0
answers
34
views
Integral of normal likelihood and multivariate normal prior
I'm updating cluster assignments in the context of a non-parametric Bayesian mixture model. When computing the probability of starting a new cluster, in the absence of cluster parameters (and using a ...
- 103
0
votes
0
answers
8
views
Choice of base measure in nCRP (for validity and computation)
I'm trying to apply the nested Chinese restaurant process (nCRP) for structure learning. nCRP is a nested extension of CRP, with each table in CRP uniquely pointing to the restaurant on the next level....
- 1
0
votes
0
answers
22
views
Relationship between Dirichlet Process and Gaussian Process
I have some questions about DP and GP.
Q1. Is there any explicit relationship between Dirichlet Process and Gaussian Process?
Q2. If there are some relationship between Gaussian and Dirichlet, can ...
- 11
2
votes
0
answers
116
views
Stick-breaking: break sticks of decreasing lengths
The stick-breaking construction used for Dirichlet Processes can create an infinite sequence of probabilities $ \boldsymbol{\pi} $ (stick lengths) that sum to 1 via the following formulae:
$\nu_i \sim ...
- 103
9
votes
4
answers
875
views
How would a bayesian estimate a mean from a large sample?
What would a bayesian do if she wanted to do inference for the mean with a large sample but has no idea of the underlying distributions?
A frequentist statitician would use the sample mean as a point ...
- 1,669
2
votes
0
answers
62
views
How to interpret this plot?
Please give me an explanation based on this
Non-linear estimation results from the aggregate model (HOSP). This figure shows the non-linear effect of age (AGE), education (EDU), family size (FAMSZ), ...
- 21
0
votes
1
answer
32
views
what does z|G mean in nonparametric bayes?
when saying z is distributed according to G, where G comes from a dirichlet process, i saw this expression:
z|G ~ G
is this same meaning with
z ~ G
?
- 3
1
vote
0
answers
88
views
Online Stochastic Variational Inference for Dirichlet Process Mixture Models
There's a 2013 NeurIPS paper I'm trying to understand, Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation. I have a few questions:
Equation 2, which defines a ...
- 935
0
votes
1
answer
132
views
Mean of draws of Dirichlet Process
Lets assume a Dirichlet process random measure in stick-breaking notation $G=\sum^\infty_{i=1} p_i \delta_{\lambda_i}$, such that $\lambda_i\sim H$ from some base distribution H, with point mass $\...
- 3
3
votes
1
answer
147
views
Why nonparametric maximum likelihood of mixture is convex
Consider $x_i \sim N(\mu_i, 1)$ where $i = 1, \ldots, n$ and assume $\mu_i$ is generated i.i.d. from an unknown distribution $F$. We are interested in estimating the unknown $\mu_i$. One way to solve ...
- 75
2
votes
1
answer
67
views
Expected Number of Dishes in Indian Buffet Process?
I'm sure this question has an answer somewhere online, but I can't find it. Suppose I have an Indian Buffet Process with $T$ customers and concentration parameter $\alpha$. For those unfamiliar with ...
- 935
3
votes
1
answer
327
views
Why does Chinese Restaurant Table Distribution look like a Gaussian Distribution?
The Chinese Restaurant Table Distribution describes the probability distribution for the number of non-empty tables in the Chinese Restaurant Process after $T$ customers have been seated. Specifically,...
- 935
2
votes
1
answer
428
views
Chinese Restaurant Process: Expected cardinality (number of customers) of each block (table)?
Short version of the question: The Chinese Restaurant Process defines a distribution over partitions of $[T] := \{1, ...., T\}$. What is the expected cardinality of the $t$th block, where $t \in \{1, ....
- 935
3
votes
0
answers
122
views
A clarification in the original Dirichlet Process paper by Ferguson
I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of ...
- 1,542
0
votes
1
answer
83
views
R fitted() function applied to rbart (Bayesian Additive Regression Trees w/ random effects) object in the dbarts package
I'm having a hard time figuring out what the output of fitted() applied to a rbart object means. Specifically, I fit my data ...
- 53
1
vote
0
answers
44
views
Dirichlet Process vs Hierarchical Dirichlet Process: coupling among transitions on infinite HMM
I'm new to nonparametric Bayesian, and I am reading a paper about beam sampling for the infinite hidden Markov model. In the paper, it is mentioned that since there is no coupling among the ...
- 11
1
vote
1
answer
160
views
What is the Dirichlet Proces Mixture Models posterior
I am trying to understand Dirichlet Process Mixture models. One of the videos I have been watching is by Tamara Broderick. I think it is a very good introductory video to Dirichlet Process mixture ...
- 1,116
3
votes
1
answer
73
views
Why are discrete random measures not dominated?
A statement I’ve seen without proof in many books and papers is that random probability measures obtained by normalizing a completely random measure, such as the Dirichlet process, are not dominated ...
- 33
3
votes
1
answer
396
views
Bayesian Nonparametric Latent feature model
For quite a long time I've been trying to understand the paper "Bayesian Nonparametric Latent feature model" (by Zoubin Ghahramani et al.) [http://mlg.eng.cam.ac.uk/zoubin/papers/GhaGriSol06.pdf].
In ...
- 325
0
votes
0
answers
44
views
Exchangabilty in CRF generative model for HDP
In the HDP Setting, the groups (or documents) are assumed exchangeable between them, and the samples (or words) within each group/topic are also exchangeable.
Note the Setting chapter here
However, ...
- 131
7
votes
1
answer
786
views
Kernel density estimate vs Dirichlet process mixture
Nowadays the Dirichlet process mixture (DPM) seems to be the default Bayesian approach for density estimation. My question is why not simply use the kernel density estimate (KDE) to model the density? ...
- 71
5
votes
1
answer
202
views
Dirichlet process mixture modelling for a Gaussian likelihood
Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
- 231
0
votes
2
answers
2k
views
Coding a simple Stick-Breaking Process in Python
I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python.
I've got this ...
- 149
10
votes
1
answer
482
views
Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?
The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a ...
1
vote
0
answers
132
views
How to interpret graphical model for Dirichlet process mixture for variational inference?
I am working through this paper by Blei and Jordan, which introduces variational inference for Dirichlet process mixtures. They derive an evidence lower bound (ELBO) function based on a stick breaking ...
- 1,025
1
vote
0
answers
75
views
Bayesian nonparametric estimate of median [closed]
I've been working on estimating the population median of a variable with a complex distribution that is not easily characterized as a parameterized probability distribution. So far, the best I have ...
- 111
2
votes
0
answers
59
views
Bayesian Model Fit for Binary Data
I am looking for methods (and preferably references) for assessing model fit for Bayesian analyses of binary data. Specifically, I am fitting Bayesian parametric and nonparametric item response models ...
- 411
5
votes
1
answer
2k
views
Gaussian Process: vector valued response
Gaussian Processes (GPs) define a prior over functions that can be updated to a posterior once we have observed data. I've been working with scalar-valued GPs, i.e. functions $f: \mathbb{R}^{d} \...
- 1,068
0
votes
1
answer
393
views
In a Dirichlet process, can the base distribution be discrete?
Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete.
1) If the base distribution is continuous, drawing from it will get a new value (a new ...
- 207
13
votes
0
answers
693
views
Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)
Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as
$$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$
Here the kernel $K$ is an assigned smooth ...
Tim♦
- 135k
0
votes
1
answer
3k
views
Normality test for likert scale [duplicate]
I have a questionnaire data that consist of composite Likert scale and discrete Likert items.
Likert scales or variables are the sum of few Likert items while the one-item variables are represented ...
3
votes
0
answers
275
views
Dirichlet Process vs. Mixture Models with Many Mixtures
The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
- 7,884
0
votes
0
answers
40
views
DPMM asymptotics for finite mixtures?
My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
- 21
2
votes
0
answers
576
views
Gibbs sampling in the Hierarchical Dirichlet Process
For an inference problem using a Dirichlet Process prior, one can derive a "basic" Gibbs sampling scheme, where we have a conditional for any parameter $\theta_i$ given the samples $x_i$ and all the ...
- 21
1
vote
0
answers
428
views
About Hierarchical Latent Dirichlet allocation (hLDA)
I am reading Blei et al.'s paper (Hierarchical Topic Models and
the Nested Chinese Restaurant Process) about hLDA.
I am confused about the details of deriving the posterior of $p(\boldsymbol c_m| \...
- 205
7
votes
2
answers
1k
views
Chinese Restaurant process (CRP)
I am trying to understand the Chinese Restaurant process (CRP) and Weighted Chinese Restaurant process (WCRP) described in a research paper "Automatic Discovery of Cognitive Skills"- Robert V. Lindsey,...
- 79
2
votes
1
answer
383
views
Question about implementing nested Chinese Restaurant Process (nCRP)
I am trying to follow the original paper on nCRP by Blei et al., 2010 and am confused with it's implementation.
The authors describe the analogy for an nCRP as follows:
A tourist arrives at the ...
- 3,482
3
votes
1
answer
222
views
Nonparametric topic modeling: hierarchical dirichlet vs. Indian buffet?
The hierarchical dirichlet process (Teh 2005) allows you to discover unlimited topics to describe a document. An alternative process, the Indian Buffet process (Griffiths 2011) is another ...
- 1,002
2
votes
1
answer
455
views
Proof of neutrality for dirichlet distribution
I am trying to learn the fields of bayesian non-parametric approaches.
I am going thru this manuscript: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf
I am bit stuck with: ...
- 237
3
votes
0
answers
98
views
Bayesian nonparametrics vs model selection using Minimum Message Length
As we know mixture models are important tools in density estimation and in general in statistical machine learning. I have always used nonparametric Bayesian mixture models to avoid the problem of ...
- 175
0
votes
1
answer
161
views
resampling hyperparameters in a Hierarchical Dirichlet Process
The sampling scheme for the hyper-parameters of hierarchical dirichlet process (HDP) is explained in the appendix of the original paper by Teh et al.
I agree that the auxiliary variable $s_j$ is a ...
- 175
9
votes
1
answer
486
views
What does it mean to integrate over a random measure?
I'm currently looking at a paper of Dirichlet process random effects model and the model specification is as follows:
$$ \begin{align*}y_{i} &= X_{i}\beta + \psi_{i} + \epsilon_{i}\\ \psi_{i} &...
- 1,142
4
votes
1
answer
617
views
Chinese Restaurant Process
I want to implement Chinese Restaurant Process representation of Dirichlet Process for random partitions. The problem setup is as follows:
I have some data (customers) which I have to randomly ...
- 201
4
votes
1
answer
198
views
What does $\in$ mean vs $=$ in probability? What does $d\phi$ mean?
In the following lecture notes on Bayesian nonparametrics http://stat.columbia.edu/~porbanz/papers/porbanz_BNP_draft.pdf, I often see something like
\begin{align}
P[\Phi_{i}\in d\phi|...]\\
P[\Phi_{i}=...
- 492
5
votes
1
answer
2k
views
Gaussian Process Regression for piecewise linear response functions
I am performing Gaussian Process Regression (without noise) for response functions which are piecewise linear.
My question: Does there exist a covariance function, such that sample paths from a ...
- 2,327
4
votes
1
answer
248
views
Marginalizing over a Chinese Restaurant Process prior
I am reading a paper by Kemp et al. and there is a part about marginalising over a Chinese Restaurant Process and I am quite clueless about how could one marginalise over such a prior! The details of ...
- 1,605
5
votes
2
answers
6k
views
Clustering methods for unknown number of clusters
Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is unknown....
- 93
1
vote
0
answers
74
views
predictive distribution of linear bayesian regression with unknow $\Sigma$ and $\Omega$
The posterior distribution for weights in linear regression setup is
\begin{equation}
\begin{aligned}
B &| Y,X \sim \mathcal{N}(\mu, \Lambda) \\
\mu &= \Lambda X^{\mathsf{T}}\Sigma^{-1}Y \\
\...
- 1,128
3
votes
1
answer
131
views
Is it possible to define the mean of a varying distribution?
Suppose $(p_1,\ldots,p_k)$ be the vector of multinomial parameters and $$(p_1,\ldots,p_k)\sim \mbox{Dirichlet}(\alpha_1,\ldots,\alpha_k).$$
Let's define a function $f(p_1,\ldots,p_k) \in \mathbb{R}$. ...
- 51
2
votes
0
answers
121
views
Proof that the Chinese restaurant process corresponds to Dirichlet process?
Let $(S, \mathcal{S})$ be a Polish space. Is there a nice proof of the fact that if the people are seated in a restaurant according to Chinese restaurant process, and then for each table, we sample a ...
- 535