Questions tagged [kernel-smoothing]
Kernel smoothing techniques, such as kernel density estimation (KDE) and Nadaraya-Watson kernel regression, estimate functions by local interpolation from data points. Not to be confused with [kernel-trick], for the kernels used e.g. in SVMs.
613
questions
1
vote
0
answers
23
views
Strong consistency of kernel density estimator
I am studying the book Nonparametric and Semiparametric Models written by Wolfgang Hardle and have difficulty with the following exercise:
$\textbf{Exercise 3.13}$ Show that $\hat{f_h}^{(n)}(x) \...
- 11
0
votes
0
answers
17
views
KDE-like technique to learn a continuous distribution from samples subject to specific noise
There's a continuous-valued random variable $X$ with distribution $f_X$. Normally, we're given a bunch of i.i.d. samples $X_1, \ldots, X_n$, and we try to give an estimate $\hat{f}_X$ of the ...
- 421
0
votes
0
answers
21
views
Kernel Density Estimation, Bandwidth Tuning, Independence, and Comparison
like many of us here, I turn to kernel density estimation when I need a nonparametric estimate of a numerical feature's distribution, and in an attempt to assume as little as possible, I usually use ...
- 121
1
vote
0
answers
35
views
Sampling subsets of a given PDF with controllable sum and frequency
I am working with a dataset with $n$ samples and $d$ features for each sample.
I would like to be able to sample "nice" subsets of this dataset with specific properties. Assume that this ...
- 11
0
votes
0
answers
53
views
How do you chose the standard deviation for a Gaussian kernel in KDE? [duplicate]
The routine explanation of a KDE plot is:
You chose a kernel (let's say a Gaussian)
You center a kernel at each data point
You average all kernels
I get that (2) means that a Gaussian curve is "...
- 240
0
votes
1
answer
12
views
How should I parse the definition of a regular kernel?
I was reading a paper on kernel regression, and the paper defines a non-negative kernel as regular if there exist $b > 0$ and $r > 0$, such that:
(1) $K(x) \ge b I\{x \in S_{0,r}\}$
(2) $\int \...
- 1
1
vote
1
answer
88
views
Kernel Smoothing for Time Series data
I have generated a time series data set of measurements that are a bit noisy and I want to apply kernel smoothing to the data. My time series data is not regular however, meaning that the time ...
- 111
0
votes
0
answers
18
views
Comparing models with transformation from discrete to continuous
I have two models to fit a set of categorical features.
One uses an encoding followed by a Kernel Density Estimation (with cross-validated bandwidth search) to make a continuous distribution. I am ...
- 101
4
votes
3
answers
765
views
How to interpret peaks in probability density function?
If a probability density function (created using kernel density estimation) exhibits peaks (not necessarily the mode), can we infer the presence of clusters or subgroups in the data?
- 105
2
votes
1
answer
48
views
Gronwall's inequality
I am reading the article.
I am getting stuck with the first proof proposition 4 on page 32.
To be more specific, they understood the reason why they obtained $F(x) \le \frac{2K}{1-\frac{2R\epsilon}{\...
- 21
1
vote
1
answer
88
views
kernel density estimation on 2D data with rotational symmetry
My question is: what is the appropriate way to apply a kernel density estimator (KDE) to a 2D dataset that has a rotational symmetry?
Specifically, I have the points ($x_i$, $y_i$) and want the ...
0
votes
0
answers
71
views
How to transform histogram to kernel density?
I have data aggregated as a histogram
$$
(m_1, c_1), (m_2, c_2), \dots, (m_k, c_k)
$$
where $m_1 < m_2 < \dots < m_k$ are the midpoints of the histogram bins and $c_i$ are the counts that sum ...
Tim♦
- 134k
1
vote
0
answers
35
views
Why is Rectangular density kernel not cut off at tails?
When we create kernel densities we could use different kernels. Here I create an example with Gaussian, Rectangular and Triangular kernel:
When we check the start and end points of the distributions ...
- 407
0
votes
1
answer
44
views
Implementing (R-)ALoKDE algorithm for data streams density estimation
I'm trying to implement the (R-)ALoKDE algorithm for the density estimation of the data streams. The algorithm has been published and presented in [1, 2]. Although the algorithm seems simple, I'm ...
0
votes
0
answers
37
views
Adaptive (variable) bandwidth kernel density estimation for univariate data
To overcome some problems with a global single bandwidth on my data with the R base function density(), I would like to try out adaptive (aka variable) bandwidths. ...
- 4,515
1
vote
0
answers
118
views
PSM kernel matching and bandwith: what observations are used
I'm using PSM with an Epanechnikov Kernel and a bandwith of 0.06.
I'm confused about which observations are matched. I thought it was (broadly) like a wheighted radius matching, where every control ...
- 11
1
vote
1
answer
39
views
How are these simulated sample means created/plotted in R?
I had a student point out this image in Learning Statistics with Jamovi that is also in Learning Statistics with R (Page 294 in the latter). I was going to send her a reply when I noticed something in ...
- 6,011
8
votes
1
answer
167
views
Radial axis transformation in polar kernel density estimate
Consider a kernel density estimate of a continuous, non-negative random variable defined over the unit circle with no discontinuity between 360 and 0 degrees.
Unlike in the most common KDE ...
- 203
4
votes
1
answer
125
views
Proving that the bias of the derivative of Parzen-Rosenblatt (kernel density) estimator is of order $O(h^2) $ and $O(h)$ when $h$ approaches $0$
I came across this property that I don't get and I couldn't find the proof anywhere:
Suppose we have a density $K$ of the standard normal distribution and $K'$ its derivative. Suppose that the density ...
- 241
1
vote
0
answers
110
views
Bias of kernel density estimator
This question is related to a calculation on page 20 of Kernel Smoothing by Wand and Jones.
i) Let $f:\mathbb{R}\to \mathbb{R}$ be a density for a real-valued random variable, and assume that $f''$ ...
- 352
1
vote
0
answers
59
views
Methods describe the temporal consistency of kernel density data
I am working on a spatial time series analysis project. The task is to study the spatial distribution of point features (e.g., crime events, traffic accidents) over time. I aim to find the places with ...
- 11
0
votes
1
answer
27
views
I can't understand how this function works (linear filter with a filter kernel)
on page 13 of the Dayan and Abbott book onTheoretical Neuroscience, there is this formula
$$r_{approx}(t)= \int\limits_{-\infty}^\infty{d\tau w(\tau) \rho(t-\tau)}$$
Let's assume that $w(t)$ is a ...
- 547
1
vote
0
answers
45
views
Are low-rank kernel approximations implementing implicit regularization?
Consider a kernel estimation problem as follows. We have functions $f^* \sim GP(0, C^*)$ drawn from a Gaussian process. We want to construct a kernel $K$ that does well in regressing functions drawn ...
- 173
2
votes
1
answer
80
views
Reason why kernel density graphs are so different in Python versus R
I plotted the same data in R using geom_density, but the blip for "Yes" is much, much smaller in Python using kdeplot ...
1
vote
0
answers
98
views
Propensity score non parametric estimation
In several papers, in the 'double machine learning' literature, the propensity score (a nuisance parameter) is estimated non parametrically. It is a bit unclear how this estimation is performed, as ...
- 101
1
vote
1
answer
129
views
Generate a point cloud distributed according to a Boltzmann-Gibbs distribution with prescribed marginals
Let $p$ be a probability density on $\Omega\in\mathcal B(\mathbb R^d)$ for some $d\in\mathbb N$ (I'm primarily interested in $\Omega=[0,1)^d$). We can approximate $p$ by $$A_x(y):=\sum_{i=1}^k\varphi_{...
- 71
0
votes
0
answers
55
views
Storing a probability distribution without saving single values
I saw this question "Storing a probability distribution without saving single values" on stackexchange and thought it deserved a statistical answer.
Example Scenario
I could see this problem ...
- 1,921
1
vote
0
answers
62
views
Gasser Müller estimator for estimating the derivative $m'(x)$ of a nonparametric regression function
I would like to compare the performance of the Gasser Müller estimator with other estimators for estimating the the derivative $m'(x)$ of the regression function $m(x)$.
Let's say we have the ...
- 111
1
vote
0
answers
25
views
Maximum bias for NW estimator when $r(x)$ is Lipschitz (question 17, chapter 5 All of Non-Parametric Statistics)
The general condition is that $Y_i = r(X_i) + \epsilon_i$, and we want to estimate $r$ using Nadaraya–Watson kernel regression.
We additionally assume $r\colon [0,1] \to \mathbb{R}$ is lipschitz, so $|...
- 554
1
vote
0
answers
147
views
Bias of kernel density estimator of pdf $f$, where $f$ has bounded first derivative $f'$
Let's say the kernel density estimator is given by
$$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\frac{X_i-x}{h_n}\right),$$ where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability ...
- 554
0
votes
0
answers
37
views
Kernel Density Estimator: Misunderstanding in Taylor Series and the bias of KDE [duplicate]
Let's say the kernel density estimator is given by
$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K(\frac{X_i-x}{h_n})$, where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability distribution ...
- 554
2
votes
1
answer
258
views
my density/kde plots don't show what I expected
I thought I understood kde/density plots until this problem. I have a dataset with two columns, Diff and Var with 5 million rows ...
- 485
1
vote
0
answers
101
views
Plotting Kernel Density Estimates on Same Axes in R
Here:
https://www.python-graph-gallery.com/74-density-plot-of-several-variables
Python's seaborn library is used to illustrate how to plot multiple kernel density ...
- 1,433
1
vote
0
answers
27
views
Closeness of two estimators of median under non parametric setup in a large sample situation
Median Regression under non-parametric set-up (Nadaraya Watson Estimate)
Data: $\{(Y_i,X_i):1\le i\le n\}$
Interested in estimating $\phi(x)=\text{median}(Y|X=x).$
Possible estimates are
Minimize the ...
- 187
3
votes
1
answer
282
views
KDE bandwidth estimation in R and Python
I am trying to estimate the bandwidth parameter of a multivariate KDE in R and then use the estimate to evaluate the KDE in Python.
The reason for this somewhat convoluted approach is that my project ...
- 314
1
vote
1
answer
64
views
Using KDE to approximate a Price vs Quantity curve
I am trying to approximate Price-vs-Quantity (P-Q) curve of a dynamic product (think Hotels, Airlines etc). As you can imagine, if you take a hotel property, the price of rooms (assuming the same room ...
- 123
1
vote
1
answer
132
views
How the kernel regression works?
I am working on propensity score matching using the Kernel Nadaraya–Watson kernel regression. But I am looking to understand the logic of estimation;
First, we estimate the Kernel density of each unit ...
1
vote
1
answer
109
views
How to estimate the conditional probability p(y|x) if y and x are both continuous but y is discrete given x?
For example, $P(Y=f_1(x)|X=x)=g_1(x)$, $P(Y=f_2(x)|X=x)=1-g_1(x)$. (The functions f1,f2 are unknown and need to be learned.) How can I estimate such a conditional probability?
I guess that kernel ...
- 11
2
votes
0
answers
158
views
Question regarding Kernel Density Estimation bandwidth selection (Scott's rule)
I'm studying KDE and got trouble understanding Scott's rule or Silverman's rule for bandwidth selection.
I saw that the optimal bandwidth is the value that minimizes Mean Integrated Squared Error (...
- 43
1
vote
0
answers
42
views
Mathematical underpinnings of variations on Silverman's bandwidth
When using a Gaussian kernel to estimate the distribution of a Gaussian-distributed $x$, the bandwidth that minimizes the mean integrated squared error is:
$$h=\left(\frac{4 \hat{\sigma}^5}{3n}\right)^...
- 140
0
votes
1
answer
276
views
How to convert units of KDE into physical units
I have a sample of people's heights. If I histogram it, I would usually divide by the bin width, so that my result becomes independent of my bin width, and I would label the axis something like (...
- 173
0
votes
0
answers
54
views
Compromise between ranked and kernel-based estimate of empirical distribution (for estimating likelihood of single value)
Setup
I have:
About fifty values $x_1,…,x_n$ sampled from the same but unknown distribution $X$. I have no useful theoretical insight on the nature of that distribution. Empirically, the distribution ...
- 2,261
3
votes
0
answers
405
views
Pros and cons of Nadaraya–Watson estimator vs. RKHS method?
Recently I've been reading some materials about nonparametric methods. Two methods related to the word "kernel" rasied my interest-- Nadaraya–Watson estimator and RKHS method.
What's the ...
- 31
0
votes
1
answer
188
views
Spatial analysis / kernel density in R, what kernel describes this distribution of points?
So, I have a scenario where I want to model the probability distribution / density of random displacement using kernel density, but having trouble finding resources on how the math works. I'm working ...
0
votes
0
answers
39
views
How to prove symmetry of a Uniform kernel?
I am trying to prove this kernel is valid,
$$
K(x) = \frac{1}{2}I(-1 < x < 1)
$$
So far I can integrate to 1, but how do I prove $$k(x) = k(-x)$$
Also, how do we satisfy that k(x) is $\ge$ 0 for ...
1
vote
0
answers
45
views
Kernel Density: How do the terms 'global' and 'pilot' translate?
I nearly most of the articles on kernel smoothing or concepts that use kernel density estimations, authors speak of 'pilot' and 'global'.
https://link.springer.com/article/10.1023/A:1008925425102
&...
- 141
2
votes
1
answer
479
views
Is Kernel-Regression parametric or non-parametric?
As the title says, is kernel regression a parametric or non-parametric method, and how can this be motivated/explained?
- 35
1
vote
1
answer
71
views
How to efficiently bias future samples?
My goal is to hit all of my 'targets' through a random variable $Y : [0,1] \rightarrow \mathbb{R}$. An explicit form of $Y$ is unknown but I am able to take a sample point as pass it to $Y$ and get ...
- 71
1
vote
0
answers
148
views
Intuition for bandwidth and degrees of freedom in kernel smoothers
For Kernel smoothers such as local polynomial regression smoothers (the Nadaraya-Watson smoother), we consider $y = m(x) + \epsilon$, for $m(x)$ some smooth function we are trying to estimate and $\...
- 421
1
vote
0
answers
135
views
Kernel Density Estimation using a Two-Boundary-Kernel à la Jones
I'm trying to understand how to perform a KDE on a bounded support, i.e. with lower and upper boundary, when using a kernel that is specifically designed to ensure consistency/$h^2$-bias at the ...
- 11