Questions tagged [nonparametric-bayes]

Bayesian methods for infinite dimensional parameter spaces.

Filter by
Sorted by
Tagged with
4
votes
1answer
57 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
0
votes
0answers
11 views

Is Laud & Ibrahim's (1995) D(m) statistic a better model-fit indicator than the log-likelihood?

I am comparing model fit between a parametric (P) item response model and a Bayesian nonparametric (BNP) item response model. The log-likelihoods for the BNP model are more extreme than those for the ...
0
votes
1answer
139 views

Coding a simple Stick-Breaking Process in Python

I've just red the great 2012 blog post of Edwin Chen about Dirichlet Process with companion code in R and Ruby. Then I'm trying to translate the Stick-Breaking Process from R to Python. I've got this ...
9
votes
1answer
152 views

Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?

The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a ...
0
votes
0answers
16 views

Derivation of prior for non-exchangeable Indian Buffet Process

Can anyone tell me how one derives to this probability mass function for any Z generated by the non-exchangeable IBP: $$ \mathbb{P}(Z) = \frac{\alpha^{K_+}}{\prod_{i=1}^N K_1^{(i)}!} \exp(-\alpha H_N)...
0
votes
0answers
77 views

Nonparametric Bayesian estimation of several black-box functions of different variables from their noisy sums

In order to introduce my problem, let’s start with the nonparametric estimation of a single unknown/black-box function $f:{\Omega _f} \to \mathbb{R}$ of a discrete variable $x$ in a finite domain ${\...
1
vote
0answers
48 views

How to interpret graphical model for Dirichlet process mixture for variational inference?

I am working through this paper by Blei and Jordan, which introduces variational inference for Dirichlet process mixtures. They derive an evidence lower bound (ELBO) function based on a stick breaking ...
4
votes
1answer
427 views

Gaussian Process: vector valued response

Gaussian Processes (GPs) define a prior over functions that can be updated to a posterior once we have observed data. I've been working with scalar-valued GPs, i.e. functions $f: \mathbb{R}^{d} \...
0
votes
1answer
155 views

In a Dirichlet process, can the base distribution be discrete?

Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete. 1) If the base distribution is continuous, drawing from it will get a new value (a new ...
8
votes
0answers
216 views

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth ...
0
votes
1answer
535 views

Normality test for likert scale [duplicate]

I have a questionnaire data that consist of composite Likert scale and discrete Likert items. Likert scales or variables are the sum of few Likert items while the one-item variables are represented ...
3
votes
0answers
142 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
0
votes
0answers
31 views

DPMM asymptotics for finite mixtures?

My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
2
votes
0answers
308 views

Gibbs sampling in the Hierarchical Dirichlet Process

For an inference problem using a Dirichlet Process prior, one can derive a "basic" Gibbs sampling scheme, where we have a conditional for any parameter $\theta_i$ given the samples $x_i$ and all the ...
1
vote
0answers
364 views

About Hierarchical Latent Dirichlet allocation (hLDA)

I am reading Blei et al.'s paper (Hierarchical Topic Models and the Nested Chinese Restaurant Process) about hLDA. I am confused about the details of deriving the posterior of $p(\boldsymbol c_m| \...
7
votes
2answers
738 views

Chinese Restaurant process (CRP)

I am trying to understand the Chinese Restaurant process (CRP) and Weighted Chinese Restaurant process (WCRP) described in a research paper "Automatic Discovery of Cognitive Skills"- Robert V. Lindsey,...
1
vote
1answer
270 views

Question about implementing nested Chinese Restaurant Process (nCRP)

I am trying to follow the original paper on nCRP by Blei et al., 2010 and am confused with it's implementation. The authors describe the analogy for an nCRP as follows: A tourist arrives at the ...
1
vote
1answer
132 views

Nonparametric topic modeling: hierarchical dirichlet vs. Indian buffet?

The hierarchical dirichlet process (Teh 2005) allows you to discover unlimited topics to describe a document. An alternative process, the Indian Buffet process (Griffiths 2011) is another ...
1
vote
1answer
187 views

Proof of neutrality for dirichlet distribution

I am trying to learn the fields of bayesian non-parametric approaches. I am going thru this manuscript: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf I am bit stuck with: ...
3
votes
0answers
86 views

Bayesian nonparametrics vs model selection using Minimum Message Length

As we know mixture models are important tools in density estimation and in general in statistical machine learning. I have always used nonparametric Bayesian mixture models to avoid the problem of ...
0
votes
1answer
97 views

resampling hyperparameters in a Hierarchical Dirichlet Process

The sampling scheme for the hyper-parameters of hierarchical dirichlet process (HDP) is explained in the appendix of the original paper by Teh et al. I agree that the auxiliary variable $s_j$ is a ...
9
votes
1answer
159 views

What does it mean to integrate over a random measure?

I'm currently looking at a paper of Dirichlet process random effects model and the model specification is as follows: $$ \begin{align*}y_{i} &= X_{i}\beta + \psi_{i} + \epsilon_{i}\\ \psi_{i} &...
4
votes
1answer
261 views

Chinese Restaurant Process

I want to implement Chinese Restaurant Process representation of Dirichlet Process for random partitions. The problem setup is as follows: I have some data (customers) which I have to randomly ...
3
votes
1answer
180 views

What does $\in$ mean vs $=$ in probability? What does $d\phi$ mean?

In the following lecture notes on Bayesian nonparametrics http://stat.columbia.edu/~porbanz/papers/porbanz_BNP_draft.pdf, I often see something like \begin{align} P[\Phi_{i}\in d\phi|...]\\ P[\Phi_{i}=...
4
votes
1answer
760 views

Gaussian Process Regression for piecewise linear response functions

I am performing Gaussian Process Regression (without noise) for response functions which are piecewise linear. My question: Does there exist a covariance function, such that sample paths from a ...
2
votes
1answer
128 views

Marginalizing over a Chinese Restaurant Process prior

I am reading a paper by Kemp et al. and there is a part about marginalising over a Chinese Restaurant Process and I am quite clueless about how could one marginalise over such a prior! The details of ...
5
votes
2answers
3k views

Clustering methods for unknown number of clusters

Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is unknown....
1
vote
0answers
66 views

predictive distribution of linear bayesian regression with unknow $\Sigma$ and $\Omega$

The posterior distribution for weights in linear regression setup is \begin{equation} \begin{aligned} B &| Y,X \sim \mathcal{N}(\mu, \Lambda) \\ \mu &= \Lambda X^{\mathsf{T}}\Sigma^{-1}Y \\ \...
3
votes
1answer
69 views

Is it possible to define the mean of a varying distribution?

Suppose $(p_1,\ldots,p_k)$ be the vector of multinomial parameters and $$(p_1,\ldots,p_k)\sim \mbox{Dirichlet}(\alpha_1,\ldots,\alpha_k).$$ Let's define a function $f(p_1,\ldots,p_k) \in \mathbb{R}$. ...
1
vote
0answers
93 views

Proof that the Chinese restaurant process corresponds to Dirichlet process?

Let $(S, \mathcal{S})$ be a Polish space. Is there a nice proof of the fact that if the people are seated in a restaurant according to Chinese restaurant process, and then for each table, we sample a ...
0
votes
1answer
348 views

Likely mean of a multinomial distribution with dirichlet prior

I am working to create a Bayesian non-parametric estimate of the mean of a distribution given a distribution of observations. Ultimately I'd like to get to a credibility interval of the likely mean of ...
1
vote
1answer
230 views

For inference of Dirichlet Process Mixture, why the expected value $\int h(x)f(x)$ is desired?

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture? $f(x)$ is the probability density function, ...
1
vote
0answers
36 views

Is it a problem to have non homogeneus sampling time in Bayes Filter?

I have a doubt related with Recursive State Estimation using Bayes Filter (actually using an aproximation to that through Particle Filters) This algorithm is explained in several sources with ...
22
votes
3answers
1k views

Is there a Bayesian approach to density estimation

I am interested to estimate the density of a continuous random variable $X$. One way of doing this that I learnt is the use of Kernel Density Estimation. But now I am interested in a Bayesian ...
0
votes
1answer
1k views

Understanding the difference between Supervised and unsupervised learning?

I have been reading about the Supervised and Unsupervised learning. What I came to know through this link is that in case of Supervised learning you have a set of input and a set of labels which are ...
7
votes
3answers
850 views

Books for learning non parametric Bayesian model

Having studied parametric Bayesian statistics during the two last years, I plan to begin to self-study non parametric Bayesian model during this summer and look for recommendations. I would like the ...
5
votes
1answer
1k views

understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for ...
4
votes
1answer
3k views

stick breaking model of Dirichlet process

I have a question regarding sticking-breaking model of Dirichlet process, which is defined as follows: There are further statements that I am not clear that how to derive equation 1 from that ...
1
vote
0answers
29 views

Is there Bayesian search theory about search through the space of Bayesian models?

Consider we start with a specific Bayesian model, say an infinite mixture model with a Dirichlet Process as a prior. I know there are wildly many variants on this theme, from the Hierarchical ...
1
vote
0answers
54 views

How well can the Dirichlet process cluster really small datasets?

I have been debating between a model-based parametric clustering approach (e.g. HMMs), and a hierarchical Dirichlet/Pitman-Yor process for clustering sequential data. I understand the latter has been ...
1
vote
1answer
316 views

How to draw samples from a Bayesian nonparamatric density estimation? [DPpackage]

I am trying to compute a Kernel Density from high dimensional data ($n > 2$). The underlying (generative) model is assumed unknown. The goal is to draw samples from this estimate, in a sense ...
1
vote
1answer
312 views

Why semi/nonparametric models?

Increasing the flexibility of models makes it prone to overfitting. On the other hand, it looks to me that, if the space function classes $\mathcal{F}$ is too big, it is hard to prove bounds on ...
10
votes
2answers
5k views

PyMC for nonparametric clustering: Dirichlet process to estimate Gaussian mixture's parameters fails to cluster

Problem setup One of the first toy problems I wanted to apply PyMC to is nonparametric clustering: given some data, model it as a Gaussian mixture, and learn the number of clusters and each cluster's ...
0
votes
0answers
103 views

How to sample via blocked Gibbs the conjugate normal model?

i am trying to sample the two dimensional parameter $\theta=(\mu,\sigma^2)$, for the Normal model. I have the full conditionals being for the mean: $\pi(\mu_{j}|\ldots) \sim N\left(\frac{\xi_{j}\...
10
votes
4answers
8k views

Gaussian Processes: How to use GPML for multi-dimensional output

Is there a way to perform Gaussian Process Regression on multidimensional output (possibly correlated) using GPML? In the demo script I could only find a 1D example. A similar question on CV that ...
1
vote
0answers
178 views

In GPML, why does scaler in Squared Exponent covariance function gets very small?

I'm running regression using GPML with a covariance function being a sum of a Gaussian noise and a Squared Exponent (SE). Input is in R4 and both the input and the output are normalized. I run ...
3
votes
0answers
213 views

Estimating parameters of categorical distribution from sum-of-outcome data

Let $X$ be a categorical random variable with possible outcomes $o_1,...,o_n \subset [l, u]$ (real numbers with a known lower bound $l>0$ and known upper bound $u$) that occur with probability $p(X ...
3
votes
0answers
905 views

Non-parametric estimate of conditional expectation

I have a (fairly smooth) function $f$ and a sample $\{(x_i,y_i)\}_{i=1,\ldots,N}$ from the joint distribution of the random variables $X$ and $Y$. I would like to estimate the conditional expectation ...
2
votes
0answers
152 views

Points to keep in mind while implementing a nonparametric bayesian inference procedure from scratch

I have been trying to implement a Bayesian inference procedure from scratch for a specific problem, but I have implemented the procedure, and it doesn't seem to work. Since, I can't just post the ...
9
votes
1answer
1k views

Putting a prior on the concentration parameter in a Dirichlet process

Most of this is background, skip to the end if you already know enough about Dirichlet process mixtures. Suppose I am modeling some data as coming from a mixture of Dirichlet processes, i.e. let $F \...