Questions tagged [density-estimation]
Estimation of probability density functions, whether by kernel density estimation, log-spline estimation or other methods.
324
questions
1
vote
0
answers
25
views
Why is histogram density estimation nonparametric?
My understanding of histogram density estimation:
For $k$ predefined equal-width bins $(b_0, b_1], (b_1, b_2], ..., (b_{k-1}, b_k]$ and $n$ observations $x_1,...,x_n \in (b_0,b_k]$, we estimate ...
- 11
0
votes
2
answers
43
views
Flexmix maxima are not where they are expected to be
For my dataset I have plotted the density with ggplot.
As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus.
...
- 111
2
votes
0
answers
40
views
Estimating the distribution of a sum of two random variables if the family of one of the variables is known
Assume I have a random variable $Y=X_1+X_2$. I want to estimate the distribution $f$ of $Y$ given a sample $y_1,\ldots,y_N$. If this was all that is known about $Y$ the best way would probably be to ...
- 2,641
0
votes
0
answers
36
views
Density estimation for time series data
Suppose I have a collection of time series for a number of subjects, say $y_{ij}$ are measurements for subject $i$ at time $t_{ij}$. The times are not uniformly sampled and each subject may have a ...
- 492
0
votes
0
answers
13
views
Population Estimation And Conditional Probability
Let's say that I have a data set comprised of age data for a large number of individuals, as well as a unique identifier for each individual. For terminology sake let's call this my base population. ...
4
votes
3
answers
238
views
Parametric models for mixed discrete/continuous data
I'm curious if there are any common parametric distribution models for mixed discrete/continuous data. For illustration, suppose I have two random vectors, $X_c,X_d$, where $X_c$ is continuous and $...
- 492
2
votes
1
answer
83
views
Loss function for estimating the conditional variance by fitting $y_i^2$
I'm trying to detect anomolies in a dataset $i \in \{1,2,...,N\}$ where a random variable $y_i$ is expected to be drawn from a normal distribution with mean $\mu_i=0$ and variance $\sigma_i^2 (X_i)$ ...
- 187
0
votes
0
answers
53
views
Expected value (and variance) of a Dirichlet Process
Suppose I have a measure $G$ that follows a Dirichlet Process,
$$G \sim DP(H_0,\alpha)$$
where $H_0$ is some base measure. Is there a closed form solution for the expected value of $G$?
1
vote
1
answer
88
views
kernel density estimation on 2D data with rotational symmetry
My question is: what is the appropriate way to apply a kernel density estimator (KDE) to a 2D dataset that has a rotational symmetry?
Specifically, I have the points ($x_i$, $y_i$) and want the ...
5
votes
2
answers
440
views
Is density estimation the same as parameter estimation?
I was studying parameter estimation from Sheldon Ross' probability and statistics book. Here the task of parameter estimation is described as follows:
Is this task the same of density estimation in ...
- 151
0
votes
0
answers
71
views
How to transform histogram to kernel density?
I have data aggregated as a histogram
$$
(m_1, c_1), (m_2, c_2), \dots, (m_k, c_k)
$$
where $m_1 < m_2 < \dots < m_k$ are the midpoints of the histogram bins and $c_i$ are the counts that sum ...
Tim♦
- 134k
1
vote
0
answers
35
views
Why is Rectangular density kernel not cut off at tails?
When we create kernel densities we could use different kernels. Here I create an example with Gaussian, Rectangular and Triangular kernel:
When we check the start and end points of the distributions ...
- 407
1
vote
0
answers
46
views
Computation of conditional expectation [closed]
Suppose that we have a random vector $X \in \Bbb R^n$ and a random variable $Y \in \Bbb R$, and that the joint density $f(x, y)$ is known. For a given $x \in \Bbb R^n$, what is the most efficient way ...
- 41
2
votes
0
answers
48
views
Package for Multidimensional Density Estimation
I may be missing something obvious, but is there a python package that can reliably do density estimation of a PDF in high dimensions (e.g. 512)? I know of scipy's ...
- 21
0
votes
0
answers
39
views
Sample from one distribution such that it’s PDF matches another distribution
Problem: I have a set of samples from a continuous distribution (multivariate), call this set $W$. I have another set of samples from a different distribution $X$. I want to sample from $W$ (with ...
- 21
0
votes
0
answers
31
views
Building a mixed-categorical-continuous probability distribution
I have a variable space that consists of some mixed categorical, integer, and real valued variables i.e $X = (x_1, ..., x_n), x_1 \in \mathbb{Z}, x_2 \in \mathbb{R}, x_n \in \{1,2\}$, and a (small) ...
- 135
1
vote
1
answer
40
views
Online Estimation of a Joint Distribution from batches of data
I want to implement an algorithm for the online estimation of a joint probability distribution from a sequence of mini batches sampled from the real distribution. The distribution is discrete and non ...
- 11
0
votes
0
answers
19
views
Output of EBM energy network
I implemented an energy-based model (EBM) as the prior of a generative model. The energy network is a small multi-layer perceptron with parameters $\mathbf{\theta}$. The prior probability is given as
$...
- 1
0
votes
0
answers
55
views
Storing a probability distribution without saving single values
I saw this question "Storing a probability distribution without saving single values" on stackexchange and thought it deserved a statistical answer.
Example Scenario
I could see this problem ...
- 1,921
1
vote
0
answers
104
views
How to quantitatively compare parametric density fit and kernel density (KDE) fit of a multivariate data?
I am working on modeling the joint distribution of given multivariate data. I can fit some parametric distributions on the data and evaluate the fitted models by LogLiklihood and AIC values. However, ...
- 11
0
votes
0
answers
30
views
Meshfree vs grid discretisation of a probability density
Let $\mu$ be a probability distribution on $\mathbb{R}^d$ with density $f$. Suppose I cannot sample from $f$.
I am coding an algorithm that involves the unknown density $f$, of course I need ...
1
vote
0
answers
149
views
Bias of kernel density estimator of pdf $f$, where $f$ has bounded first derivative $f'$
Let's say the kernel density estimator is given by
$$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\frac{X_i-x}{h_n}\right),$$ where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability ...
- 554
0
votes
0
answers
37
views
Kernel Density Estimator: Misunderstanding in Taylor Series and the bias of KDE [duplicate]
Let's say the kernel density estimator is given by
$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K(\frac{X_i-x}{h_n})$, where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability distribution ...
- 554
1
vote
1
answer
55
views
Density Estimation of a Matrix-valued Random Variable?
It seems like the density estimation of a multivariate vector-valued random variable has been well studied, but what if one would like to estimate the probability density of a matrix-valued random ...
- 13
1
vote
0
answers
24
views
Normalized Density vs Unormalized Density [duplicate]
Edited: I have been researching about density function estimation from a sample of data, and I noticed that there are a lot of researches the estimate the density with the normalizing factor and ...
- 63
0
votes
0
answers
36
views
How are probability density functions, that are computed from real-world datasets, stored and represented by computational software?
In probability and statistics, density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density ...
- 373
2
votes
0
answers
19
views
Is it correct to say 'estimate of probability density function'?
This question is about terminology:
I have a stochastic process from which I get a sample. Ideally I want to know the probability density function (pdf) associated with the process, but from the data ...
- 591
1
vote
0
answers
54
views
Density Estimation of High-dimensional Data
I would like to estimate the probability density function of a data set with a very large number of samples (50,000+) and a large number of continuous variables (2,048).
Compute efficiency is somewhat ...
4
votes
0
answers
109
views
How to fit a copula when zeros abound?
I am modelling a joint distribution for two random variables: $F(x,y)$. I observe $n$ data points $(x^{}_{i},y^{}_{i})^{N}_{i=1}$. I would like to model $F$ as the product of its marginals and a ...
- 93
1
vote
1
answer
350
views
Fitting a copula vs. directly fitting a multivariate distribution
I understand that the joint density of two random variables $f(x,y)$ can be decomposed as the product of its marginals and a copula: $f(x,y) = g(x)k(y) \times c(G(x),K(y))$. Alternatively this may be ...
- 93
1
vote
1
answer
110
views
How to estimate the conditional probability p(y|x) if y and x are both continuous but y is discrete given x?
For example, $P(Y=f_1(x)|X=x)=g_1(x)$, $P(Y=f_2(x)|X=x)=1-g_1(x)$. (The functions f1,f2 are unknown and need to be learned.) How can I estimate such a conditional probability?
I guess that kernel ...
- 11
0
votes
0
answers
39
views
How to prove symmetry of a Uniform kernel?
I am trying to prove this kernel is valid,
$$
K(x) = \frac{1}{2}I(-1 < x < 1)
$$
So far I can integrate to 1, but how do I prove $$k(x) = k(-x)$$
Also, how do we satisfy that k(x) is $\ge$ 0 for ...
4
votes
1
answer
450
views
Estimate parameters of an unknown negative binomial distribution based on known distribution
The PDF of a known NBD given in Equation (1). The parameter a and r are function of $μ$ = sample mean, and $s^2$ = sample variance, as given in Equation (2) and (3) respectively. $r$ = number of ...
- 241
1
vote
0
answers
46
views
Universal Approximation Capabilities of Mixture of Weibulls
Can a mixture of $N$ Weibull distributions approximate any continuous density with non-negative support, if $N$ is sufficiently large? (If so, a reference to the proof would be greatly appreciated).
(...
- 115
1
vote
0
answers
45
views
Kernel Density: How do the terms 'global' and 'pilot' translate?
I nearly most of the articles on kernel smoothing or concepts that use kernel density estimations, authors speak of 'pilot' and 'global'.
https://link.springer.com/article/10.1023/A:1008925425102
&...
- 141
1
vote
0
answers
119
views
How to understand the density in machine learning?
We can calculate the conditional density using Eq.1[3].
$$
p_{\theta, \Lambda}(y \mid \boldsymbol{x})=\frac{\exp \left(f_{\theta, \Lambda}(\boldsymbol{x})[y]\right)}{\sum_{k=1}^{n} \exp \left(f_{\...
- 11
1
vote
0
answers
135
views
Kernel Density Estimation using a Two-Boundary-Kernel à la Jones
I'm trying to understand how to perform a KDE on a bounded support, i.e. with lower and upper boundary, when using a kernel that is specifically designed to ensure consistency/$h^2$-bias at the ...
- 11
14
votes
3
answers
871
views
Kernel Density Estimate for Cauchy
As far as I understand, kernel density estimation does not make any assumptions on the moments of the underlying density, and just requires smoothness. The Cauchy density function is quite smooth. ...
- 15k
0
votes
0
answers
109
views
Approximation of a polynomial via histogram
Note: I originally tried to pose this question generally, without discussing the specific type of stochastic process. I hope that this can still be an interesting question generally.
Assume that we ...
1
vote
0
answers
194
views
How to sample from a distribution approximated by a Neural Network?
There are a few models already that approximate distributions with a neural network i.e.: energy models define a density function $f(x)= e^{S(x,w)}/Z$ where $S$ is a neural network and $Z$ is a ...
- 633
1
vote
1
answer
403
views
Bandwidth Selection for Kernel Density Estimation
Are there any heuristics for selecting the bandwidth for kernel density estimation? In other words, is a spiky curve better or a smooth one?
- 37
1
vote
0
answers
68
views
I have difficulties to understand the input normalization - density estimation in the same context in ML or DL
I know (by experimenting with different ML and DL algorithms) that input normalization helps to improve the performance of the model. When we do normalization in training, with the same mean and ...
- 171
1
vote
0
answers
69
views
Optimal rate of convergence of nonparametric density estimators
Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
- 140
0
votes
0
answers
337
views
Is possible to compare two density distributions 'trends'?
I have responses from two groups (A and B) on a confidence rating about a distance judgment task. The participants saw pairs of stimuli and after they were asked to rate their confidence about the ...
- 11
0
votes
0
answers
14
views
Density estimation for 50 input variables - individually or together
I am a beginner to statistics and was wondering if it is possible to estimate the density where there are 50 input variables.
Do I perform density estimation for each variable individually then ...
- 143
1
vote
0
answers
82
views
Expectation of kernel density estimators using sharpened data
My question regards the proof of the bias of the kernel density estimator obtained using "sharpened" data. The method comes from the paper by Choi and Hall (1999).
Specifically, assume $X_1, ...
- 545
1
vote
0
answers
36
views
Driver based forecasting using past distributions
I have reduced my original forecasting problem (Short context : I need to forecast hotel bookings and checkins for the next 3 months. I already have a reasonable forecast for bookings and need to ...
4
votes
5
answers
203
views
Determine traffic speed distribution while driving
I assume that the vehicle speed of cars on a highway is normally distributed around the posted speed limit. I could verify this by sitting by the road with a radar gun and measure vehicle speed for a ...
1
vote
0
answers
23
views
How is Dowd's (2007) resampling procedure supposed to mitigate the problem of autocorrelated multiple-step-ahead forecasts?
Dowd "Validating multiple‐period density‐forecasting models" (2007) considers evaluation of multiple-step-ahead density forecasts. There is a know problem of dependence between forecast ...
- 63.6k
1
vote
3
answers
1k
views
Practical Uses of Kernel Density Estimators
Perhaps this question is too broad, but I would like to know - how does one use a kernel density estimate in practice? I know of course that one can use it to draw pretty pictures on top of histograms,...
- 115