Questions tagged [kernel-smoothing]
Kernel smoothing techniques, such as kernel density estimation (KDE) and Nadaraya-Watson kernel regression, estimate functions by local interpolation from data points. Not to be confused with [kernel-trick], for the kernels used e.g. in SVMs.
622
questions
2
votes
0
answers
97
views
Textbook Recommendation other than ESL [duplicate]
My current background is as follows: (core subjects only)
Math : Linear Algebra, Analysis, (half of) Measure TheoryStats : Mathematical Statistics, Regression Analysis, Multivariate Analysis
"...
- 217
0
votes
0
answers
30
views
Why are my KDE log-densities so high? [closed]
I fitted a gaussian KDE to 512-dimensional data using sklearn's KernelDensity. I sampled from the resulting model and calculated the log-densities of the samples using the score_samples method. ...
- 13
3
votes
2
answers
87
views
Positive Semidefinite Kernel in RKHS
The following shows part of the page 170 of The Element of Statistical Learning that I want to make clear.
The solution can be characterized in two equivalent ways
$$\min_{c_j}\sum_{i=1}^N(y_i - \...
- 217
1
vote
1
answer
71
views
Regularization Problem and Reproducing Kernel Hilbert Space
The following shows part of the page 169 of The Element of Statistical Learning that I want to make clear.
We have
$$\min_{f \in \mathcal H_K}[\sum_{i = 1}^NL(y_i, f(x_i)) + \lambda\Vert f\Vert_{\...
- 217
0
votes
0
answers
21
views
Is polynomial interpolation with RKHS in some way more advantageous than simple Lagrange interpolation?
The reproducing kernel Hilbert space associated with the polynomial kernel $K(x,z)=(1+xz)^{d-1}$ (or other similar polynomials) can be used to interpolate a continuous function $f$ at by its value at ...
- 163
0
votes
1
answer
42
views
Addition of asymmetrical uncertainties for use in KDE?
My data have asymmetrical confidence intervals (e.g. 100 with a 95% lower bound of 50 and 95% upper bound of 300). I want to perform a KDE on all my data where the bandwidth is determined by the ...
- 1
1
vote
0
answers
10
views
Kernel density estimation for noisy samples with known non-iid noise
I'm interested in the following variant of the usual one-dimensional density-estimation problem:
I wish to estimate some unknown density $\rho$. There are iid samples $Y_{1},\ldots,Y_{n} \sim \rho$, ...
- 11
1
vote
0
answers
27
views
Kernel Density Estimation on a Log-Scale: Log Transformation vs. Geometric Space
I’m working on a project where I need to plot a Kernel Density Estimation (KDE) on a log-scale x-axis. I’ve come across two different methods and I’m unsure which one would be more appropriate for my ...
- 11
4
votes
1
answer
381
views
Difference between KDE, MLE and EM for density estimation
I'm reviewing kernel density estimation (KDE), maximum likelihood estimation (MLE) and expectation maximization (EM) algorithm for density estimation and struggling to differentiate what each ...
- 143
1
vote
0
answers
25
views
Strong consistency of kernel density estimator
I am studying the book Nonparametric and Semiparametric Models written by Wolfgang Hardle and have difficulty with the following exercise:
$\textbf{Exercise 3.13}$ Show that $\hat{f_h}^{(n)}(x) \...
- 11
0
votes
0
answers
24
views
KDE-like technique to learn a continuous distribution from samples subject to specific noise
There's a continuous-valued random variable $X$ with distribution $f_X$. Normally, we're given a bunch of i.i.d. samples $X_1, \ldots, X_n$, and we try to give an estimate $\hat{f}_X$ of the ...
- 421
0
votes
0
answers
23
views
Kernel Density Estimation, Bandwidth Tuning, Independence, and Comparison
like many of us here, I turn to kernel density estimation when I need a nonparametric estimate of a numerical feature's distribution, and in an attempt to assume as little as possible, I usually use ...
- 121
1
vote
0
answers
36
views
Sampling subsets of a given PDF with controllable sum and frequency
I am working with a dataset with $n$ samples and $d$ features for each sample.
I would like to be able to sample "nice" subsets of this dataset with specific properties. Assume that this ...
- 11
0
votes
0
answers
128
views
How do you chose the standard deviation for a Gaussian kernel in KDE? [duplicate]
The routine explanation of a KDE plot is:
You chose a kernel (let's say a Gaussian)
You center a kernel at each data point
You average all kernels
I get that (2) means that a Gaussian curve is "...
- 240
0
votes
1
answer
15
views
How should I parse the definition of a regular kernel?
I was reading a paper on kernel regression, and the paper defines a non-negative kernel as regular if there exist $b > 0$ and $r > 0$, such that:
(1) $K(x) \ge b I\{x \in S_{0,r}\}$
(2) $\int \...
- 1
1
vote
1
answer
184
views
Kernel Smoothing for Time Series data
I have generated a time series data set of measurements that are a bit noisy and I want to apply kernel smoothing to the data. My time series data is not regular however, meaning that the time ...
- 111
0
votes
0
answers
18
views
Comparing models with transformation from discrete to continuous
I have two models to fit a set of categorical features.
One uses an encoding followed by a Kernel Density Estimation (with cross-validated bandwidth search) to make a continuous distribution. I am ...
- 101
4
votes
3
answers
883
views
How to interpret peaks in probability density function?
If a probability density function (created using kernel density estimation) exhibits peaks (not necessarily the mode), can we infer the presence of clusters or subgroups in the data?
- 125
2
votes
1
answer
52
views
Gronwall's inequality
I am reading the article.
I am getting stuck with the first proof proposition 4 on page 32.
To be more specific, they understood the reason why they obtained $F(x) \le \frac{2K}{1-\frac{2R\epsilon}{\...
- 21
1
vote
1
answer
108
views
kernel density estimation on 2D data with rotational symmetry
My question is: what is the appropriate way to apply a kernel density estimator (KDE) to a 2D dataset that has a rotational symmetry?
Specifically, I have the points ($x_i$, $y_i$) and want the ...
0
votes
0
answers
89
views
How to transform histogram to kernel density?
I have data aggregated as a histogram
$$
(m_1, c_1), (m_2, c_2), \dots, (m_k, c_k)
$$
where $m_1 < m_2 < \dots < m_k$ are the midpoints of the histogram bins and $c_i$ are the counts that sum ...
Tim♦
- 136k
1
vote
0
answers
45
views
Why is Rectangular density kernel not cut off at tails?
When we create kernel densities we could use different kernels. Here I create an example with Gaussian, Rectangular and Triangular kernel:
When we check the start and end points of the distributions ...
- 359
0
votes
1
answer
46
views
Implementing (R-)ALoKDE algorithm for data streams density estimation
I'm trying to implement the (R-)ALoKDE algorithm for the density estimation of the data streams. The algorithm has been published and presented in [1, 2]. Although the algorithm seems simple, I'm ...
0
votes
0
answers
57
views
Adaptive (variable) bandwidth kernel density estimation for univariate data
To overcome some problems with a global single bandwidth on my data with the R base function density(), I would like to try out adaptive (aka variable) bandwidths. ...
- 4,672
1
vote
0
answers
129
views
PSM kernel matching and bandwith: what observations are used
I'm using PSM with an Epanechnikov Kernel and a bandwith of 0.06.
I'm confused about which observations are matched. I thought it was (broadly) like a wheighted radius matching, where every control ...
- 11
1
vote
1
answer
46
views
How are these simulated sample means created/plotted in R?
I had a student point out this image in Learning Statistics with Jamovi that is also in Learning Statistics with R (Page 294 in the latter). I was going to send her a reply when I noticed something in ...
- 8,665
8
votes
1
answer
179
views
Radial axis transformation in polar kernel density estimate
Consider a kernel density estimate of a continuous, non-negative random variable defined over the unit circle with no discontinuity between 360 and 0 degrees.
Unlike in the most common KDE ...
- 203
4
votes
1
answer
143
views
Proving that the bias of the derivative of Parzen-Rosenblatt (kernel density) estimator is of order $O(h^2) $ and $O(h)$ when $h$ approaches $0$
I came across this property that I don't get and I couldn't find the proof anywhere:
Suppose we have a density $K$ of the standard normal distribution and $K'$ its derivative. Suppose that the density ...
- 241
1
vote
0
answers
163
views
Bias of kernel density estimator
This question is related to a calculation on page 20 of Kernel Smoothing by Wand and Jones.
i) Let $f:\mathbb{R}\to \mathbb{R}$ be a density for a real-valued random variable, and assume that $f''$ ...
- 352
1
vote
0
answers
61
views
Methods describe the temporal consistency of kernel density data
I am working on a spatial time series analysis project. The task is to study the spatial distribution of point features (e.g., crime events, traffic accidents) over time. I aim to find the places with ...
0
votes
1
answer
29
views
I can't understand how this function works (linear filter with a filter kernel)
on page 13 of the Dayan and Abbott book onTheoretical Neuroscience, there is this formula
$$r_{approx}(t)= \int\limits_{-\infty}^\infty{d\tau w(\tau) \rho(t-\tau)}$$
Let's assume that $w(t)$ is a ...
- 557
1
vote
0
answers
50
views
Are low-rank kernel approximations implementing implicit regularization?
Consider a kernel estimation problem as follows. We have functions $f^* \sim GP(0, C^*)$ drawn from a Gaussian process. We want to construct a kernel $K$ that does well in regressing functions drawn ...
- 173
2
votes
1
answer
93
views
Reason why kernel density graphs are so different in Python versus R
I plotted the same data in R using geom_density, but the blip for "Yes" is much, much smaller in Python using kdeplot ...
2
votes
0
answers
117
views
Propensity score non parametric estimation
In several papers, in the 'double machine learning' literature, the propensity score (a nuisance parameter) is estimated non parametrically. It is a bit unclear how this estimation is performed, as ...
- 111
1
vote
1
answer
132
views
Generate a point cloud distributed according to a Boltzmann-Gibbs distribution with prescribed marginals
Let $p$ be a probability density on $\Omega\in\mathcal B(\mathbb R^d)$ for some $d\in\mathbb N$ (I'm primarily interested in $\Omega=[0,1)^d$). We can approximate $p$ by $$A_x(y):=\sum_{i=1}^k\varphi_{...
- 141
0
votes
0
answers
71
views
Storing a probability distribution without saving single values
I saw this question "Storing a probability distribution without saving single values" on stackexchange and thought it deserved a statistical answer.
Example Scenario
I could see this problem ...
- 1,921
1
vote
0
answers
94
views
Gasser Müller estimator for estimating the derivative $m'(x)$ of a nonparametric regression function
I would like to compare the performance of the Gasser Müller estimator with other estimators for estimating the the derivative $m'(x)$ of the regression function $m(x)$.
Let's say we have the ...
- 111
1
vote
0
answers
29
views
Maximum bias for NW estimator when $r(x)$ is Lipschitz (question 17, chapter 5 All of Non-Parametric Statistics)
The general condition is that $Y_i = r(X_i) + \epsilon_i$, and we want to estimate $r$ using Nadaraya–Watson kernel regression.
We additionally assume $r\colon [0,1] \to \mathbb{R}$ is lipschitz, so $|...
- 606
1
vote
0
answers
183
views
Bias of kernel density estimator of pdf $f$, where $f$ has bounded first derivative $f'$
Let's say the kernel density estimator is given by
$$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\frac{X_i-x}{h_n}\right),$$ where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability ...
- 606
0
votes
0
answers
37
views
Kernel Density Estimator: Misunderstanding in Taylor Series and the bias of KDE [duplicate]
Let's say the kernel density estimator is given by
$\hat f(x) = \frac{1}{nh_n} \sum_{i=1}^n K(\frac{X_i-x}{h_n})$, where $h_n \to 0$, $nh_n \to \infty$, $K$ a symmetric probability distribution ...
- 606
2
votes
1
answer
349
views
my density/kde plots don't show what I expected
I thought I understood kde/density plots until this problem. I have a dataset with two columns, Diff and Var with 5 million rows ...
- 485
1
vote
0
answers
115
views
Plotting Kernel Density Estimates on Same Axes in R
Here:
https://www.python-graph-gallery.com/74-density-plot-of-several-variables
Python's seaborn library is used to illustrate how to plot multiple kernel density ...
- 1,537
1
vote
0
answers
29
views
Closeness of two estimators of median under non parametric setup in a large sample situation
Median Regression under non-parametric set-up (Nadaraya Watson Estimate)
Data: $\{(Y_i,X_i):1\le i\le n\}$
Interested in estimating $\phi(x)=\text{median}(Y|X=x).$
Possible estimates are
Minimize the ...
- 197
3
votes
1
answer
326
views
KDE bandwidth estimation in R and Python
I am trying to estimate the bandwidth parameter of a multivariate KDE in R and then use the estimate to evaluate the KDE in Python.
The reason for this somewhat convoluted approach is that my project ...
- 314
1
vote
1
answer
65
views
Using KDE to approximate a Price vs Quantity curve
I am trying to approximate Price-vs-Quantity (P-Q) curve of a dynamic product (think Hotels, Airlines etc). As you can imagine, if you take a hotel property, the price of rooms (assuming the same room ...
- 123
1
vote
1
answer
164
views
How the kernel regression works?
I am working on propensity score matching using the Kernel Nadaraya–Watson kernel regression. But I am looking to understand the logic of estimation;
First, we estimate the Kernel density of each unit ...
1
vote
1
answer
122
views
How to estimate the conditional probability p(y|x) if y and x are both continuous but y is discrete given x?
For example, $P(Y=f_1(x)|X=x)=g_1(x)$, $P(Y=f_2(x)|X=x)=1-g_1(x)$. (The functions f1,f2 are unknown and need to be learned.) How can I estimate such a conditional probability?
I guess that kernel ...
- 11
2
votes
0
answers
179
views
Question regarding Kernel Density Estimation bandwidth selection (Scott's rule)
I'm studying KDE and got trouble understanding Scott's rule or Silverman's rule for bandwidth selection.
I saw that the optimal bandwidth is the value that minimizes Mean Integrated Squared Error (...
- 43
1
vote
0
answers
46
views
Mathematical underpinnings of variations on Silverman's bandwidth
When using a Gaussian kernel to estimate the distribution of a Gaussian-distributed $x$, the bandwidth that minimizes the mean integrated squared error is:
$$h=\left(\frac{4 \hat{\sigma}^5}{3n}\right)^...
- 140
0
votes
1
answer
332
views
How to convert units of KDE into physical units
I have a sample of people's heights. If I histogram it, I would usually divide by the bin width, so that my result becomes independent of my bin width, and I would label the axis something like (...
- 173