Questions tagged [kernel-smoothing]

Kernel smoothing techniques, such as kernel density estimation (KDE) and Nadaraya-Watson kernel regression, estimate functions by local interpolation from data points. Not to be confused with [kernel-trick], for the kernels used e.g. in SVMs.

Filter by
Sorted by
Tagged with
1
vote
0answers
11 views

A comparison of the global optimal binwidth and local optimal binwidth of the histogram estimator

Suppose we have $X_1, \dots, X_n$ to be an i.i.d sample with unknown pdf $f(x)$ and cdf $F(x)$, and define $\hat{f} (x)$ to be the histogram estimator. We also define its Mean Integrated Square Error ...
0
votes
2answers
28 views

Gaussian Kernel density estimation by hand

I'm trying to understand the logic behind kernel density estimation. I found the explanation in wikipedia very useful, but I'm not capable yet, of having a full understanding of this method so I want ...
5
votes
0answers
81 views
+50

Minimizing MISE to find consistent estimator

Consider kernel regression estimation of the mean function $m$ of the process $$y_t = m(x_t) + \epsilon_t,$$ where $\epsilon_t$' s are correlated with covariance function $R(s,t) = \exp \{-\lambda|s-...
0
votes
1answer
17 views

Kernel density, why does my subset appear to have a larger spread than the original series?

I have a series that is 1500 observations long called alt_intercept. From it, I created a subset that contains values only if another series (called pvalue) is less ...
0
votes
1answer
30 views

why does y axis sometimes change from normal histogram to kernel density?

Consider the distributions I have plotted below. They are of the same variables, one in normal histogram form and another in kernel density (Epachanov). As far as I know, the auc of the kernel ...
0
votes
2answers
30 views

Kernel Density Estimation - Physical Interpretation?

I just read this article about the motivation for KDE. From what I understand, you are using Gaussian probability density distributions for each datapoint and then, depending on the selected kernel ...
0
votes
0answers
17 views

How can I estimate bivariate probability density for support restricted data?

I have a bivariate sample with the following kernel density estimation The issue is that there is actually a cutoff for log(Age) at about 2.5, so value greater than 2.5 has probability 0. The fitted ...
0
votes
0answers
23 views

Transformation of a random variable and KDEs — when is reweighting needed?

Suppose I have been given some data $X$ that I wish to resample according to their empirical distribution. For whatever reason, I decided to transform these variables to some other space $Y = f(X)$ ...
0
votes
0answers
57 views

How to choose sample size from probability density for computing mutual information based on continuous variables

I need to compute mutual information gain based two continuous variables $X$ and $Y$ $I(X|Y) = \int_X\int_Y p_{x.y}(x,y) \log(\frac{p_{x.y}(x,y)}{p_{x}(x)p_{y}(y)})$. I have used Kernel Density ...
1
vote
0answers
42 views

ksmooth function in R

I'm trying to understand how ksmooth function in R works (I haven't really taken much statistics other than an introductory one at college, so sorry if this is a ...
0
votes
1answer
42 views

Find PDF(X,Y) from PDF(X) and PDF(Y)

given that X and Y are not mutually exclusive, is there anyway to calculate PDF(X,Y) from PDF(X) and PDF(Y)? Following are a few plots made from the dataset. In above image i have to find how PDF(11,...
0
votes
0answers
23 views

Reading kernel distribution plot vs typical histogram

Tasked with showing the distribution of a certain data set in a different way, I wanted to try to plot a kernel density. After seeing it however, my co-worker advised against it saying that because ...
0
votes
0answers
13 views

What does alpha in smoothing stand for

I want to apply Gaussian smoothing to a dataset and came across the smth.gaussian function in R. That besides the numerical input data requires two parameters: ...
0
votes
0answers
22 views

ROC curve with symmetrical kernel

I am trying to use kernels with ROC curve, and 'm succeed to plot them but now my query is about theoretical grounds, i.e. its bias, var, etc. I want to evaluate the theorems (1 & 2) in Pulit (...
0
votes
1answer
24 views

Kernel Density estimation with guassian kernel function why don't have indicator function like other kernels [closed]

Kernel Density estimation with guassian kernel function why don't have indicator function like other kernels.
2
votes
1answer
38 views

When to use non-parametric regression such as kernel, generalized additive model, spline, and polynomial?

I understand that kernel regression is a form of non-linear/non-parametric regression. However, I know you can also use generalized additive models for non-linear regression, as well as polynomials ...
3
votes
3answers
238 views

Python: “Normalizing” kde, so it always lines up with histogram

In Python, I am attempting to find a way to plot/rescale kde's so that they match up with the histograms of the data that they are fitted to: The above is a nice example of what I am going for, but ...
0
votes
0answers
18 views

How to calculate Kernel Density for Bootstrap Likelihood

I am attempting to write R code to generate bootstrap likelihood as described in section 3 of this paper https://arxiv.org/pdf/1510.07287.pdf. I am confident that I performed the bootstraps correct, ...
0
votes
0answers
25 views

Reproducing kernels: how do I numerically compute the decomposition?

Suppose I'm given a kernel, $$ K(x,y) : \mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R} $$ In order to describe/understand the (unique) associated RKHS, I seek its eigenfunctions, as per Mercer'...
0
votes
0answers
47 views

KDE that better-preserves percentile distributions

My understanding is that a Gaussian KDE, because the kernel is symmetric, preserves the statistical mean of a distribution. For my particular case, I'd really prefer a KDE that preserved the median, ...
1
vote
1answer
61 views

Density Estimation and Data Normalization

Is there any problem to first normalize data (for example, min-max one) then use kernel density estimation to get pdf of each sample? Thanks.
0
votes
0answers
25 views

Every point has the same probability?

I am reading "Pattern recognition and machine learning" by Cristopher Bishop. In Chapter 2.5.1 "Kernel density estimator", there is written that: Let us suppose that observations are being drawn ...
0
votes
1answer
309 views

How to choose the bandwidth of a KDE in python

Python's Sklearn module provides methods to perform Kernel Density Estimation. One of the challenges in Kernel Density Estimation is the correct choice of the kernel-bandwidth. I have come across ...
0
votes
0answers
15 views

Group comparison for bivariate distributions

For two groups A and B that consist of n and m individual samples. Each individual sample has a unique 2-dimensional joint probability density functions (PDFs)of two variables. These PDFs are ...
1
vote
1answer
62 views

Estimating the population median from a kernel density estimator

I have a 1-d kernel density estimate in the form of two vectors: x_grid is a vector of x-values at which the density function was sampled ...
0
votes
2answers
86 views

Can Kernel Density Estimation estimate an Exponential Distribution?

Can Kernel Density Estimation estimate an Exponential Distribution? I tried to performed to make experiments with various kernels like: "gaussian" and "exponential", but performance seems to be very ...
0
votes
1answer
138 views

Isn't kernel ridge regression supposed to be the mean of gaussian processes?

I read a few times that the mean prediction of a GP should be equivalent to KRR. I tested this empirically and found (dataset is y=2x + gaussian noise): Two explanations for this come to mind: GP is ...
0
votes
0answers
23 views

bandwidth setting for density comparison

I intend to compare a list of density distributions. Following one published paper (with similar type of data and same objective), I learned that I have to use Gaussian kernel density estimator, and a ...
0
votes
0answers
43 views

Derivation of AMISE and Bandwidth

Given: Let $K(\cdot)$ be a bona fide kernel. Let $f$ be a pdf and $\widehat{f}_n$ is kernel density estimator with bandwidth $h$ based on a sample $X_1,X_2,\cdots,X_n$ of size $n$ draw iid from $f$. ...
2
votes
0answers
14 views

Contribution of a predictor in Nonparametric regression

Is there an equivalent to a beta weights in a nonparametric regression? I am using the NP package in R and running a local linear regression where my bandwidth estimates are produced using least ...
3
votes
1answer
57 views

Properties of Kernel Density Estimators

Given Let $X \in \mathbb{R}$ be a real-valued random variable with theoretical probability density function (pdf) $f(x)$ and corresponding cumulative distribution function (cdf) $F(x)$. Let $X_1, X_2,...
1
vote
0answers
47 views

GARCH model with t-innovations

I am modelling a time series with GARCH model with t-distributed error using RUGARCH package. My model is specified as: ...
1
vote
0answers
47 views

Adaptive Parzen Estimator

I am trying to implement the Tree Parzen Estimator for hyperparameter optimization. I follow this paper https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf, in which ...
0
votes
0answers
83 views

Confidence Region of a multivariate KDE in Python?

I have an estimated bivariate kernel density based on a set of observations (𝑥11,𝑥12,...,𝑥1𝑛) and (𝑥21,𝑥22,...,𝑥2𝑛) and would like to draw confidence regions in the (𝑥1,𝑥2) space. This is in ...
0
votes
0answers
8 views

Comparison of Kernel Home-Range Utilisation Distribution between samples with unequal numbers of locations

I have a dataset containing GPS coordinates for a sample of animals in the same location at the same time (think of an aerial snapshot of a cows in a field), at two different dates. I have estimated ...
1
vote
0answers
38 views

How to perform MCMC integration when no prior over the integrated function is available? [closed]

As far as I can tell, MCMC integration (e.g. VEGAS) is performed by sampling from a distribution proportional to $f(x)$ using MCMC, then building a density estimator $g(x)$ using these samples (for ...
0
votes
0answers
27 views

Mairhuber-Curtis Theorem, Adaptive Bases and Neural Networks

I am reading the paper Positive Definite Kernels: Past, Present and Future by GE Fasshauer in which he describes computing with PD kernels. In section 3.2 he mentions the Mairhuber-Curtis Theorem, ...
1
vote
0answers
52 views

Is There an Optimal Bandwidth for Kernel Density Estimation at One Point

I am trying to estimate the density of a 2D distribution using KDE, but I only care about the accuracy right at the origin. I have read about methods to estimate the optimal bandwidth matrix when you ...
1
vote
0answers
92 views

Kernel density estimation (KDE) for data points with different variance

Consider the following situation: An experiment was done in 15 different conditions, and a value of the parameter 'A' was measured in each experiment (A can have any integer value between 0 and ...
0
votes
0answers
54 views

Data Normalization for Time Series Forecasting using Nonparametric Model

What are the specialities of applying locally weighted learning for time series forecasting? I am trying to apply a nonparametric model ($K$-NN Regression) to forecast daily load curve (entire time ...
0
votes
0answers
24 views

How to decide the weight of the locally logistic regression?

I have a problem of how to decide the weight of the logistic regression. My model is as below. For the general logistic regression, we have the likelihood of the model: $L(\theta)=\prod_{i=1}^{m}P ( ...
1
vote
2answers
223 views

How to compute Integrated Squared Error for kernel density estimation in R

I am working on R for kernel density estimation. I am testing different kernels and I need to evaluate them. I use next code for density estimation: ...
1
vote
1answer
74 views

downsampling a kde / combining kde and histogram

I'm calculating a KDE of one parameter (y, particle density) in bins of another parameter (x, distance from the origin). At ...
0
votes
1answer
19 views

Function dependent to nth order on a sample

From this paper: ... every multivariate density estimator that is in any reasonable sense nonparametric may be written in the form: $$ \hat{f}(y) = \frac{1}{n}\sum_{i=1}^n K_n(x_i, y) $$ ...
1
vote
0answers
26 views

Hypothesis testing in non-parametric regression

Say I have two processes/time series, $X = (X_{t_{1}},X_{t_{2}},\dots , X_{t_{n}})$ and $Y = (Y_{t_{1}},Y_{t_{2}},\dots , Y_{t_{n}})$ observed at times $t_i$ for $i=1,2,\dots, n$ where $0 < t_1 <...
0
votes
0answers
44 views

Equivalent Kernel - Bishop Chapter 3

I've been struggling to understand the Equivalent Kernel in Bishop's Pattern Recognition and Machine Learning book. Can somebody explain the following Figure (3.10) from chapter 3.3.3? (image taken ...
0
votes
0answers
36 views

Bilinear Process

how to generate a functional bilinear process such as: $ X_{n+1}= \int\psi(t,s)X_{n}(s)ds + \iint\phi(t,s,u)X_{n}(s)\varepsilon_{n}(u)dsdu + \varepsilon_{n+1}(t) $ ? About the first integral there ...
0
votes
0answers
58 views

Are vanishing bias and variance enough for pointwise consistency for KDE-based estimation?

Question: Is the condition that asymptotic bias and asymptotic variance goes to zero for infinite samples sufficient to guarantee the pointwise consistency of an estimator based on plug-in kernel ...
8
votes
2answers
805 views

What did Silverman (1981) mean by 'critical bandwidth'?

In the selection of a bandwidth for a Kernel Density Estimator, critical bandwidth according to my understanding is: "For every integer k, where 1<k<n, we ...
1
vote
0answers
74 views

Can kernel density estimation be used to estimate a radial probability density?

I am trying to estimate the PDF of the radius of points distributed in a 2D plane. The points are distributed like this: I can produce a histogram of the radius data which looks like this: I want ...