Questions tagged [kernel-smoothing]

Kernel smoothing techniques, such as kernel density estimation (KDE) and Nadaraya-Watson kernel regression, estimate functions by local interpolation from data points. Not to be confused with [kernel-trick], for the kernels used e.g. in SVMs.

Filter by
Sorted by
Tagged with
1
vote
2answers
32 views

what are the Kernels with zero variance (in KDE)?

I am studying about Kernels in Kernel Density Estimation and I came to understand that the bigger the $n$ that satisfies $\int_{-\infty}^{\infty}K(u)\cdot u^{j}du=0$ for all $1\leq j \leq n$ the more ...
0
votes
0answers
36 views

Time Varying Coefficient Model with Uniform Kernel and Spline Estimator

I'm working on the BMACS data set data(BMACS) from library(npmlda). I'm looking at the the time-varying coefficient model of post-CD4 versus smoking $X_1$, pre-HIV CD4 percent $X_2$ (centered) and age ...
0
votes
0answers
7 views

Why can't I do a ksmooth function in R without a y value? [closed]

I have a x vector, containing 10 values. x=c(56,44,79,70,54,94,85,33,65,57) ksmooth(x,ker="parzen",bandwidth =11, x.points=c(50)) I am trying to find ...
1
vote
0answers
14 views

KDE Classifier: Mean of the bandwidths of features produces better results than a bandwidth for each feature. Why?

I'm building a Univariate Bayesian Kernel Density Estimation Classifier. It works like the Naives Bayes Classifier but instead of using the normal density function, it uses the Kernel Density ...
0
votes
1answer
43 views

How to clean data to produce a smooth histogram

Is there any way to 'clean' a set of data to produce a smooth histogram, ie. without overrepresentative bins? Looking for solution in python3. At the moment I have a histogram with overcounted bins: ...
1
vote
0answers
22 views

Probability Density Estimation vs Function Approximation [closed]

I have a function $f: \mathbb{R} \to \mathbb{R}_+$ and I would like to estimate it. The data pairs $\{(x_i, f(x_i))\}$ arrive at different times $t$. I have two questions: In this case, since the ...
0
votes
0answers
24 views

Kernel Density Estimation (KDE) vs Gaussian Process (Regression)

From the Wikipedia article on KDE there is no mention of Gaussian Processes (or GP regression.) Nonetheless, the two seem to be very closely related. Am I seeing "ghosts" or are these non-...
0
votes
0answers
12 views

histogram vs. kernel in density estimation

Assume we have a problem of estimation of a density $f(x)$ over an interval $[0, 1]$. Can a regular histogram (i.e. with equal-sized bins) be viewed as some kind of a kernel?
2
votes
0answers
14 views

Regression function in terms of a set of basis functions

Regression function in terms of a set of basis functions is given as $f(x) = \sum_{m=1}^M \beta_m h_m(x) + \beta_0$$ To estimate $\beta$ and $\beta_0$ we minimize the following expression. $$H(\beta, \...
1
vote
1answer
42 views

Density plot with epanechnikov with exceedance data

I'm trying to replicate empirical density plot from the paper "Computing Maximum Likelihood Estimates for the Generalized Pareto Distribution". The data is ...
0
votes
1answer
15 views

Why is bw.nrd0() function and bw=“nrd0” showing different results of density plot?

According to density function documentation in R (source), they use "nrd0" as default setting. However, when I tried <...
1
vote
1answer
38 views

What's the mathematical profof that kernel density estimate has the properties of a probability density?

I would like to know what is mathematical proof that kernel density has the properties of a probability density when bandwidth is greater than 0. I know that kernel and bandwidth are nonnegative and ...
3
votes
2answers
108 views

Does density() function in base R assume normality?

I'm plotting a vector as a density plot and it seems that as I increased the bandwidth, the distribution looks more and more normally distributed. Does density ...
0
votes
0answers
23 views

How to prove that fitting local logistic regression model amounts to smoothing binary indicator?

I'm working on Exercise 6.5 in the book "The Elements of Statistical Learning"(not as assignment). I'm not even sure if my understanding of the problem is correct. In the simplest case where ...
1
vote
2answers
107 views

How to show that smoothing spline fit preserves the local regression part of the fit

We need to show that a smoothing spline of $y_i$ to $x_i$ retains the local regression part of the fit. For linear regression, this problem seems trivial because it is relatively easy to move from $...
3
votes
2answers
114 views

Maximum Value of Kernel Function in ABC

Are there cases where a kernel function, must have 1 as the maximum value ?? The definition of a Kernel can be found in the following link, https://en.wikipedia.org/wiki/Kernel_(statistics)#In_non-...
2
votes
2answers
90 views

Density function estimation from histogram with infinite bin

Given a histogram such as the following Histogram 1 Bin Count −3.5 to −2.51 9 −2.5 to −1.51 32 −1.5 to −0.51 109 −0.5 to 0.49 180 0.5 to 1.49 132 1.5 to Inf 38 What would be the best approach ...
2
votes
1answer
142 views

concrete example of degenracy

I was reading on degeneracy. Can someone give concrete example of following definition. A kernel $h$ is symmetric if it is invariant under permutations of its inputs. $S_m$ be symmetric group ...
0
votes
1answer
44 views

How to generate a confidence interval for an adaboost prediction?

I have created a simple AdaBoostRegressor (sklearn) model which is trained on a feature set $X$ (house features) to predict a variable $y$ (house prices). The model can be used to create a prediction ...
0
votes
0answers
15 views

Uniform Kernel vs Histogram pdf estimator

I want to know if in general Histogram pdf estimator and the Uniform Kernel density estimator are the same, I have derived following 2 equations but still have trouble wrapping my head around the ...
0
votes
0answers
14 views

Terminology for smoothing methods: kernel density estimation, kernel smoothing, etc

I'm using what I have been referring to as 'kernel density estimation' to estimate the rates of a series of variables a, b, c from noisy observations distributed in ...
1
vote
1answer
27 views

Peaks of estimated probability distribution are always lower than those of true distribution - why?

I have some code (shown below) for sampling from and then estimating a normal mixture distribution in one dimension. When I plot the estimate (blue) against the true distribution (black) I get images ...
0
votes
0answers
13 views

Calculation of MISE for density estimation in deamer package?

The mean integrated square error of a density estimate $\hat p$ of a true density $p$ is $$\begin{equation} \begin{aligned} \text{MISE}(\hat{p}) &\equiv \mathbb{E} \Big[ \int (\hat{p}(x,\mathbf{X}...
0
votes
0answers
35 views

can I estimate density function of 2d/3d data with kernel smoothing (e.g. ks package R), or are there better estimation methods

I have a 2d matrix of positive values (non integer), where the values can be thought of intensity at an x,y coordinate indexed by the row and column. I want to estimate a density function across this ...
1
vote
0answers
32 views

Is it possible to convert a pdf obtained with density() to an ecdf?

Consider the following code that gives us (an estimate of) the pdf of a random variable $X$: X = rnorm(100,10,1) XDensity = density(X) I want to obtain the ecdf of ...
2
votes
1answer
43 views

Notation for expected value [duplicate]

I often find the following notation in ML Papers, with $k$ as kernel: $E_{x, y \sim p} [ k(x, y ) ]$ I am not familiar with it. Is it the expected value with $x$ and $y$ as a random variable with ...
1
vote
0answers
38 views

How to calculate confidence interval of the CDF of a non-normal distribution?

For a non-normal distribution, how to calculate the confidence interval of the Cumulative Distribution Function (CDF) of such distribution? Are there any approximations to calculate confidence limits ...
0
votes
0answers
15 views

Kernel Density Bandwith Estimation - Summary

I just read the fantastic answer posted by Glen_b back in 2016 on the topic of Kernel Density Bandwidth Estimation (KDBE). Due to taking classes during the COVID-19 pandemic, my knowledge of density ...
0
votes
0answers
30 views

Smooth autocorrelation estimator

I have a univariate time series that exhibits what looks like a smooth slowly decaying autocorrelation function. The dataset size is huge (~1bln observations). If I subsample the data taking each ...
0
votes
0answers
15 views

Use cases exist for the Silverman kernel

I am well aware of the usage of the Gaussian, the boxcar function, the Epanechnikov function and others for use cases as kernel density estimation, Gaussian processes and others. But I have never seen ...
0
votes
1answer
11 views

Estimated bandwidth for KDE does not allow to cluster data

I wanted to cluster one-dimentional data with kernel density estimation. I tried to count the optimal bandwidth with Silverman's rule of thumb and also using cross-validation. However, the obtained ...
1
vote
0answers
13 views

The asymptotic properties of $V$-statistic for mixing multivariate process

Suppose $\{X_t\}_{t \in \mathbb{Z}} \subseteq \mathbb{R}^d$ is a $\alpha$- or $\rho$-mixing process. Let $h (x, y) : \mathbb{R}^d \times \mathbb{R}^d \rightarrow \mathbb{R}$ be the symmetric kernel ...
1
vote
0answers
16 views

In kernel regression, what are the common theoretical motivations for using a kernel that is Lipschitz continuous?

I read a few papers on Nadaraya-Watson kernel regression in which I saw assumptions that require the kernel function being Lipschitz continuous without explanation ( and without citation of such ...
3
votes
1answer
186 views

How does GridSearchCV(KernelDensity(), Params) find the optimal bandwidth?

I wanna know How GridSearchCV works? I mean this method gives a grid interval for the optional bandwidth params = {'bandwidth': np.linspace(0.1, 1, 100)}, but how does it evaluate each bandwidth value?...
2
votes
0answers
108 views

Least Square Cross Validation for Density Estimation with Histograms

In a 1981 paper by Rudemo an easy to compute expression for the integrated squared error of a histogram relative to the true distribution is derived (Eq. 2.8 of the paper and the last equation in this ...
0
votes
0answers
29 views

Risk Expression using Nadaraya-Watson

I'm looking at the Epanechnikov kernal, $K(x)=\frac{3}{4}(1-x^2), -1 \leq x \leq 1$, and I want to know how it can be expressed in terms of risk, i.e. $bias^2 + variance$ using the Nadaraya-Watson ...
0
votes
0answers
16 views

Sampling from Nearest Neighbor Density Estimator

The KDE (with variable bandwidth) is defined as $$\hat f(x) = n^{-1}\sum_{i=1}^nh_x^{-1}K\big(h_x^{-1}(x - X_i)\big).$$ Once the density is estimated, one could sample points as follows: pick a poin ...
0
votes
0answers
47 views

Can anyone explain this adaptive kernel density estimation method?

I happened to need a reliable multivariete kernel density estimation method, and found it in this Matlab code, which turned out to work very well with my data. But I have trouble understanding its ...
0
votes
0answers
36 views

2 D density plot interpretation and manipulating the data?

I have a data frame like this: ...
0
votes
0answers
24 views

Biase of ASE estimation Kernel Regression

I'm trying to calculate the bias of the estimator $p(h)=n^{-1}\displaystyle\sum_{i=1}^{n}(Y_{j}-\hat{m}_{h}(X_{j})^{2}w(X_{j})$ of the averaged squared error. The result I find in the literature is ...
1
vote
1answer
54 views

Suppose $\widehat{m}'(x)$ is the derivative of Nadaraya-Watson estimator, can I get its uniform rate from the rate for its numerator and denominator?

Suppose $E(Y|x)=m(x)$ is the regression function that is twice differentiable, $f(x)$ is the density of $X$ that is also twice differentiable. Suppose $Y_i=m(X_i)+e_i$. $m'(x)$ is the derivative of ...
2
votes
0answers
40 views

Linear model with partially time-varying coefficients

Suppose we have a linear model with time-varying coefficients $$ y_i = x_i' \beta_{t_i} + \epsilon_i, \; i = 1, 2, \cdots, n $$ where the design points are $t_i = \frac{i}{n}$, and $\beta(t): [0,1] \...
0
votes
0answers
29 views

Why $E[E[p_{N}(Z_n,Z_m)|Z_m]]\neq E[E[p_{N}(Z_n,Z_m)|Z_n]]$ here? What went wrong?

We have a U-statistic defined as $U=\frac{1}{N(N-1)}\sum_{n\neq m}p_{N}(Z_n,Z_m)$, where observations $\{Z_i\}_{i=1}^{N}$ are i.i.d. following distribution $F(z)$ with density $f(z)$. $p_{N}(Z_n,Z_m)=\...
0
votes
1answer
31 views

Kernel density estimation - different choice of kernel

There are many choices of kernel function to use in kernel density estimation: Gaussian, Epanechnikov, Uniform, Triangular, and so on. I'm using Gaussian kernel to estimate density of two-...
1
vote
0answers
23 views

Parameter estimation for basis function model in Elements of Statistical Learning (ESL)

In the book Elements of Statistical Learning, section 2.8.3 describes Basis Functions, citing an example of a radial basis function as $f_{\theta}(x) = \sum_{m=1}^M \beta_M \sigma(\alpha_m'x + b_m)$, ...
1
vote
1answer
108 views

Why taking an average makes convergence to zero faster?

Let $f(x,y)$ be some density, and let the leave-one-out Nadaraya-Watson estimator $\widehat{f}_{-i}(x,y)$ be defined as follows: $\widehat{f}_{-i}(x,y)=\frac{1}{(n-1)h^2}\sum_{j=1,j\neq i}^nK(\frac{(...
0
votes
1answer
165 views

Gaussian and Epanechnikov Kernel Regressions giving drastically different estimations

sorry if this is the wrong place to be asking this question. I'm trying to implement kernel regression for a specific dataset I'm working with, but I'm noticing that the trendlines generated by my ...
0
votes
0answers
9 views

Simulate Random Draws From Density Estimated By a Balloon Estimator

I estimate a density using the balloon KDE $$\hat f(x) = \frac{k}{n\mathrm{vol}(S_d)r_k(x)},$$ where $\mathrm{vol}(S_d)$ denotes the volume of the $d$-dimensional unit ball and $r_k(x)$ the Euclidean ...
0
votes
1answer
16 views

smoother: “… prediction on x, which is unrelated to the values of x_i.” What does it mean?

My lecture notes says Any practical implementation of a smoother is based on input in the form of a scatterplot $(x_i, y_i)_{i=1}^n$, on a tuning parameter $h$, and on a grid of output points $x$ ...
0
votes
0answers
40 views

Empirical estimation of conditional distribution $Y|X$ at the boundaries of X

I want to estimate conditional distributions of Y | X. Where X contains several continuous covariates. I'm coding in R. I tried several methods so far, but none gives me entirely satisfactory results ...

1
2 3 4 5
12