Questions tagged [density-estimation]
Estimation of probability density functions, whether by kernel density estimation, log-spline estimation or other methods.
330
questions
3
votes
1
answer
62
views
Is this a known or valid divergence between two densities?
I am testing various metrics for learning a density estimate. Specifically, I have a sample of data from a distribution $p$, and am learning a function $f$ to estimate $p$ by minimizing a distance or ...
0
votes
0
answers
20
views
Reference datasets for conditional density estimation
[In case you feel inclined to close this question because I'm asking for a dataset - I'm looking for solutions in the spirit of point 2 (on-topic) in the accepted answer to this question about asking ...
1
vote
0
answers
20
views
Developing a Confidence Interval of Density Functions for Uniform Periods in Seasonal Time Series Data
Suppose I have a set of observational data as a time series where the observations are collected at uniform interval over the course of several years. The data exhibits seasonality over the course of ...
0
votes
0
answers
42
views
Measuring the Distance Between KDE Distributions with Different Bin Counts
I have two KDE distributions, each with a different number of bins. I'd like to compare them effectively, and I'm wondering if there's a recommended technique for this. Should I unify the number of ...
0
votes
0
answers
19
views
Is there a method to estimate the distribution of error term in linear model?
Consider the linear model where $A$ is not known
$$
y = Ax + \epsilon
$$
where we want to estimate the distribution $\epsilon$ from a set of samples. To prevent over-fitting, we want to impose some ...
1
vote
0
answers
22
views
At what circumstances will the difficulty for the tasks of density evaluation and sampling be different?
In this tutorial video of normalizing flow, the presenter mentioned that for the original autoregressive flow, the density evaluation is fast and the sampling is slow. In contrast, for the inverse ...
1
vote
0
answers
14
views
Kernel density estimation for noisy samples with known non-iid noise
I'm interested in the following variant of the usual one-dimensional density-estimation problem:
I wish to estimate some unknown density $\rho$. There are iid samples $Y_{1},\ldots,Y_{n} \sim \rho$, ...
1
vote
0
answers
57
views
Kernel Density Estimation on a Log-Scale: Log Transformation vs. Geometric Space
I’m working on a project where I need to plot a Kernel Density Estimation (KDE) on a log-scale x-axis. I’ve come across two different methods and I’m unsure which one would be more appropriate for my ...
4
votes
1
answer
511
views
Difference between KDE, MLE and EM for density estimation
I'm reviewing kernel density estimation (KDE), maximum likelihood estimation (MLE) and expectation maximization (EM) algorithm for density estimation and struggling to differentiate what each ...
1
vote
0
answers
41
views
Why is histogram density estimation nonparametric?
My understanding of histogram density estimation:
For $k$ predefined equal-width bins $(b_0, b_1], (b_1, b_2], ..., (b_{k-1}, b_k]$ and $n$ observations $x_1,...,x_n \in (b_0,b_k]$, we estimate ...
0
votes
2
answers
67
views
Flexmix maxima are not where they are expected to be
For my dataset I have plotted the density with ggplot.
As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus.
...
2
votes
0
answers
49
views
Estimating the distribution of a sum of two random variables if the family of one of the variables is known
Assume I have a random variable $Y=X_1+X_2$. I want to estimate the distribution $f$ of $Y$ given a sample $y_1,\ldots,y_N$. If this was all that is known about $Y$ the best way would probably be to ...
0
votes
0
answers
58
views
Density estimation for time series data
Suppose I have a collection of time series for a number of subjects, say $y_{ij}$ are measurements for subject $i$ at time $t_{ij}$. The times are not uniformly sampled and each subject may have a ...
0
votes
0
answers
13
views
Population Estimation And Conditional Probability
Let's say that I have a data set comprised of age data for a large number of individuals, as well as a unique identifier for each individual. For terminology sake let's call this my base population. ...
4
votes
3
answers
380
views
Parametric models for mixed discrete/continuous data
I'm curious if there are any common parametric distribution models for mixed discrete/continuous data. For illustration, suppose I have two random vectors, $X_c,X_d$, where $X_c$ is continuous and $...
2
votes
1
answer
120
views
Loss function for estimating the conditional variance by fitting $y_i^2$
I'm trying to detect anomolies in a dataset $i \in \{1,2,...,N\}$ where a random variable $y_i$ is expected to be drawn from a normal distribution with mean $\mu_i=0$ and variance $\sigma_i^2 (X_i)$ ...
0
votes
0
answers
76
views
Expected value (and variance) of a Dirichlet Process
Suppose I have a measure $G$ that follows a Dirichlet Process,
$$G \sim DP(H_0,\alpha)$$
where $H_0$ is some base measure. Is there a closed form solution for the expected value of $G$?
1
vote
1
answer
120
views
kernel density estimation on 2D data with rotational symmetry
My question is: what is the appropriate way to apply a kernel density estimator (KDE) to a 2D dataset that has a rotational symmetry?
Specifically, I have the points ($x_i$, $y_i$) and want the ...
5
votes
2
answers
520
views
Is density estimation the same as parameter estimation?
I was studying parameter estimation from Sheldon Ross' probability and statistics book. Here the task of parameter estimation is described as follows:
Is this task the same of density estimation in ...
1
vote
0
answers
101
views
How to transform histogram to kernel density?
I have data aggregated as a histogram
$$
(m_1, c_1), (m_2, c_2), \dots, (m_k, c_k)
$$
where $m_1 < m_2 < \dots < m_k$ are the midpoints of the histogram bins and $c_i$ are the counts that sum ...
1
vote
0
answers
52
views
Why is Rectangular density kernel not cut off at tails?
When we create kernel densities we could use different kernels. Here I create an example with Gaussian, Rectangular and Triangular kernel:
When we check the start and end points of the distributions ...
1
vote
0
answers
59
views
Computation of conditional expectation [closed]
Suppose that we have a random vector $X \in \Bbb R^n$ and a random variable $Y \in \Bbb R$, and that the joint density $f(x, y)$ is known. For a given $x \in \Bbb R^n$, what is the most efficient way ...
2
votes
0
answers
50
views
Package for Multidimensional Density Estimation
I may be missing something obvious, but is there a python package that can reliably do density estimation of a PDF in high dimensions (e.g. 512)? I know of scipy's ...
0
votes
0
answers
43
views
Sample from one distribution such that it’s PDF matches another distribution
Problem: I have a set of samples from a continuous distribution (multivariate), call this set $W$. I have another set of samples from a different distribution $X$. I want to sample from $W$ (with ...
1
vote
1
answer
49
views
Online Estimation of a Joint Distribution from batches of data
I want to implement an algorithm for the online estimation of a joint probability distribution from a sequence of mini batches sampled from the real distribution. The distribution is discrete and non ...
0
votes
0
answers
82
views
Storing a probability distribution without saving single values
I saw this question "Storing a probability distribution without saving single values" on stackexchange and thought it deserved a statistical answer.
Example Scenario
I could see this problem ...
1
vote
0
answers
170
views
How to quantitatively compare parametric density fit and kernel density (KDE) fit of a multivariate data?
I am working on modeling the joint distribution of given multivariate data. I can fit some parametric distributions on the data and evaluate the fitted models by LogLiklihood and AIC values. However, ...
1
vote
0
answers
219
views
Bias of kernel density estimator of pdf $f$, where $f$ has bounded first derivative $f'$
Let's say the kernel density estimator is given by
$$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\frac{X_i-x}{h_n}\right),$$ where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability ...
0
votes
0
answers
39
views
Kernel Density Estimator: Misunderstanding in Taylor Series and the bias of KDE [duplicate]
Let's say the kernel density estimator is given by
$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K(\frac{X_i-x}{h_n})$, where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability distribution ...
1
vote
1
answer
68
views
Density Estimation of a Matrix-valued Random Variable?
It seems like the density estimation of a multivariate vector-valued random variable has been well studied, but what if one would like to estimate the probability density of a matrix-valued random ...
1
vote
0
answers
24
views
Normalized Density vs Unormalized Density [duplicate]
Edited: I have been researching about density function estimation from a sample of data, and I noticed that there are a lot of researches the estimate the density with the normalizing factor and ...
0
votes
0
answers
40
views
How are probability density functions, that are computed from real-world datasets, stored and represented by computational software?
In probability and statistics, density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density ...
2
votes
0
answers
19
views
Is it correct to say 'estimate of probability density function'?
This question is about terminology:
I have a stochastic process from which I get a sample. Ideally I want to know the probability density function (pdf) associated with the process, but from the data ...
1
vote
0
answers
56
views
Density Estimation of High-dimensional Data
I would like to estimate the probability density function of a data set with a very large number of samples (50,000+) and a large number of continuous variables (2,048).
Compute efficiency is somewhat ...
4
votes
0
answers
144
views
How to fit a copula when zeros abound?
I am modelling a joint distribution for two random variables: $F(x,y)$. I observe $n$ data points $(x^{}_{i},y^{}_{i})^{N}_{i=1}$. I would like to model $F$ as the product of its marginals and a ...
2
votes
1
answer
522
views
Fitting a copula vs. directly fitting a multivariate distribution
I understand that the joint density of two random variables $f(x,y)$ can be decomposed as the product of its marginals and a copula: $f(x,y) = g(x)k(y) \times c(G(x),K(y))$. Alternatively this may be ...
1
vote
1
answer
138
views
How to estimate the conditional probability p(y|x) if y and x are both continuous but y is discrete given x?
For example, $P(Y=f_1(x)|X=x)=g_1(x)$, $P(Y=f_2(x)|X=x)=1-g_1(x)$. (The functions f1,f2 are unknown and need to be learned.) How can I estimate such a conditional probability?
I guess that kernel ...
0
votes
0
answers
46
views
How to prove symmetry of a Uniform kernel?
I am trying to prove this kernel is valid,
$$
K(x) = \frac{1}{2}I(-1 < x < 1)
$$
So far I can integrate to 1, but how do I prove $$k(x) = k(-x)$$
Also, how do we satisfy that k(x) is $\ge$ 0 for ...
4
votes
1
answer
517
views
Estimate parameters of an unknown negative binomial distribution based on known distribution
The PDF of a known NBD given in Equation (1). The parameter a and r are function of $μ$ = sample mean, and $s^2$ = sample variance, as given in Equation (2) and (3) respectively. $r$ = number of ...
1
vote
0
answers
52
views
Universal Approximation Capabilities of Mixture of Weibulls
Can a mixture of $N$ Weibull distributions approximate any continuous density with non-negative support, if $N$ is sufficiently large? (If so, a reference to the proof would be greatly appreciated).
(...
1
vote
0
answers
46
views
Kernel Density: How do the terms 'global' and 'pilot' translate?
I nearly most of the articles on kernel smoothing or concepts that use kernel density estimations, authors speak of 'pilot' and 'global'.
https://link.springer.com/article/10.1023/A:1008925425102
&...
1
vote
0
answers
142
views
How to understand the density in machine learning?
We can calculate the conditional density using Eq.1[3].
$$
p_{\theta, \Lambda}(y \mid \boldsymbol{x})=\frac{\exp \left(f_{\theta, \Lambda}(\boldsymbol{x})[y]\right)}{\sum_{k=1}^{n} \exp \left(f_{\...
1
vote
0
answers
152
views
Kernel Density Estimation using a Two-Boundary-Kernel à la Jones
I'm trying to understand how to perform a KDE on a bounded support, i.e. with lower and upper boundary, when using a kernel that is specifically designed to ensure consistency/$h^2$-bias at the ...
14
votes
3
answers
952
views
Kernel Density Estimate for Cauchy
As far as I understand, kernel density estimation does not make any assumptions on the moments of the underlying density, and just requires smoothness. The Cauchy density function is quite smooth. ...
0
votes
0
answers
122
views
Approximation of a polynomial via histogram
Note: I originally tried to pose this question generally, without discussing the specific type of stochastic process. I hope that this can still be an interesting question generally.
Assume that we ...
1
vote
0
answers
230
views
How to sample from a distribution approximated by a Neural Network?
There are a few models already that approximate distributions with a neural network i.e.: energy models define a density function $f(x)= e^{S(x,w)}/Z$ where $S$ is a neural network and $Z$ is a ...
1
vote
1
answer
529
views
Bandwidth Selection for Kernel Density Estimation
Are there any heuristics for selecting the bandwidth for kernel density estimation? In other words, is a spiky curve better or a smooth one?
1
vote
0
answers
75
views
I have difficulties to understand the input normalization - density estimation in the same context in ML or DL
I know (by experimenting with different ML and DL algorithms) that input normalization helps to improve the performance of the model. When we do normalization in training, with the same mean and ...
1
vote
0
answers
92
views
Optimal rate of convergence of nonparametric density estimators
Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
0
votes
0
answers
409
views
Is possible to compare two density distributions 'trends'?
I have responses from two groups (A and B) on a confidence rating about a distance judgment task. The participants saw pairs of stimuli and after they were asked to rate their confidence about the ...