Is there a clear analytic link from Kernel smoothing, particularly the Nadaraya–Watson estimator, (S-G, H-P, or W-H) smoothing filter?
The "filter" is called by different names in different fields including analytic chemists & engineers, economists, actuaries, and the statisticians (who tend to know several variants).
When I say "kernel method"for smoothing I am referring to an approach like this (link), or possibly this (link). I give an example in code below as well. A Gaussian kernel is a variation on the Normal probability density, which by definition is non-negative. The Epanechnikov and Uniform estimators are defined as piece-wise but share this constraint.
The "filter", like a kernel based smoother, makes a weighted sum of a window of points around the point of interest. Unlike the smoother, the "filter" has negative valued coefficients.
How do I relate the negative weights in the "filter" with a kernel approach? Is there a connection between the two? Are they fundamentally incompatible? How would I make a more robust Gaussian kernel smoother that has negative weights.
What I mean is this:
The coefficients for the 5-element quadratic smoothing filter are "-3, 12,17,12,and -3" with a normalization constant of 35. The absolute values sum to 41. This means ~15% of weight is negative. Is there a derivation from good solid first principles that gives a Normal Kernel smooth with similarly valued coefficients?
For example, let's say that I want estimate y_true from y_meas as given by the following.
set.seed(231529)
#create data
N <- 200
t <- seq(from=1,to=7*pi,length.out = N)
y_true <- 0.23*sin(1.5*t) - 0.5*cos(0.2*t+0.9)
#corrupt with noise
epsilon <- +runif(n = N,min=-0.1,max = 0.1)
y_meas <- y_true + epsilon
#plot it
plot(t,y_meas, xlab = "t", ylab="y")
lines(t,y_true,col="Green",lwd=2)
I can do this with Savitzky-Golay as follows:
#Savitzy-Golay
library(pracma)
y_sg <- savgol(y_meas, #my data
fl = 17,
dorder=0, # snoothing
forder=3) # quadratic
lines(t,y_sg,col="Red")
While the Kernel methods are a way to approximate a probability density function there is this: $$ \mu = \frac{\int x\cdot p\left(x\right) } {\int p \left(x\right) } $$ or in a discrete case $$ \mu = \frac{\Sigma x\cdot p\left(x\right) } {\Sigma p\left(x\right) } $$ and so it can make a clean weighted windowed mean for non-uniformly distributed data.
#Kernel
y_k <- numeric(length = length(t) )
for (i in 9:(N-8)){
xi <- t[(i-8):(i+8)]-t[i]; #domain
yi <- y_meas[(i-8):(i+8)]; #range
temp <- density(xi, #the window
bw = "sj",
kernel = "gaussian")
#evaluate at our points -
w <- interp1(x=temp$x,y=temp$y,xi=xi)
#normalize
w <- w/sum(w)
#find mean
y_k[i] <- sum(w*yi)
}
lines(t,y_k,col="Blue",lwd=3)
For the same input data, the SG, using "eyeball norm", seems to do better.
Here is my plot.
The difference between the two is that SG (or such) is robust (negative coefficients) in a way that the classic Gaussian Kernel is not, unless there is a good reason to adjust the weights. Because it has a window, the N-W estimator is tending to regress toward the mean and so it overshoots the valleys and undershoots the peaks.
I appreciate the responses so far, and will go dig through them. Thanks.
