Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [kernel-smoothing]

Kernel smoothing techniques, such as kernel density estimation (KDE) and Nadaraya-Watson kernel regression, estimate functions by local interpolation from data points. Not to be confused with [kernel-trick], for the kernels used e.g. in SVMs.

0
votes
1answer
11 views

Kernel Density estimation with guassian kernel function why don't have indicator function like other kernels [on hold]

Kernel Density estimation with guassian kernel function why don't have indicator function like other kernels.
2
votes
1answer
23 views

When to use non-parametric regression such as kernel, generalized additive model, spline, and polynomial?

I understand that kernel regression is a form of non-linear/non-parametric regression. However, I know you can also use generalized additive models for non-linear regression, as well as polynomials ...
2
votes
3answers
46 views

Python: “Normalizing” kde, so it always lines up with histogram

In Python, I am attempting to find a way to plot/rescale kde's so that they match up with the histograms of the data that they are fitted to: The above is a nice example of what I am going for, but ...
0
votes
0answers
6 views

How to calculate Kernel Density for Bootstrap Likelihood

I am attempting to write R code to generate bootstrap likelihood as described in section 3 of this paper https://arxiv.org/pdf/1510.07287.pdf. I am confident that I performed the bootstraps correct, ...
0
votes
0answers
23 views

Reproducing kernels: how do I numerically compute the decomposition?

Suppose I'm given a kernel, $$ K(x,y) : \mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R} $$ In order to describe/understand the (unique) associated RKHS, I seek its eigenfunctions, as per Mercer'...
0
votes
0answers
18 views

KDE that better-preserves percentile distributions

My understanding is that a Gaussian KDE, because the kernel is symmetric, preserves the statistical mean of a distribution. For my particular case, I'd really prefer a KDE that preserved the median, ...
1
vote
1answer
47 views

Density Estimation and Data Normalization

Is there any problem to first normalize data (for example, min-max one) then use kernel density estimation to get pdf of each sample? Thanks.
0
votes
0answers
23 views

Every point has the same probability?

I am reading "Pattern recognition and machine learning" by Cristopher Bishop. In Chapter 2.5.1 "Kernel density estimator", there is written that: Let us suppose that observations are being drawn ...
0
votes
1answer
71 views

How to choose the bandwidth of a KDE in python

Python's Sklearn module provides methods to perform Kernel Density Estimation. One of the challenges in Kernel Density Estimation is the correct choice of the kernel-bandwidth. I have come across ...
0
votes
0answers
15 views

Group comparison for bivariate distributions

For two groups A and B that consist of n and m individual samples. Each individual sample has a unique 2-dimensional joint probability density functions (PDFs)of two variables. These PDFs are ...
1
vote
1answer
32 views

Estimating the population median from a kernel density estimator

I have a 1-d kernel density estimate in the form of two vectors: x_grid is a vector of x-values at which the density function was sampled ...
0
votes
2answers
55 views

Can Kernel Density Estimation estimate an Exponential Distribution?

Can Kernel Density Estimation estimate an Exponential Distribution? I tried to performed to make experiments with various kernels like: "gaussian" and "exponential", but performance seems to be very ...
0
votes
1answer
53 views

Isn't kernel ridge regression supposed to be the mean of gaussian processes?

I read a few times that the mean prediction of a GP should be equivalent to KRR. I tested this empirically and found (dataset is y=2x + gaussian noise): Two explanations for this come to mind: GP is ...
0
votes
0answers
23 views

bandwidth setting for density comparison

I intend to compare a list of density distributions. Following one published paper (with similar type of data and same objective), I learned that I have to use Gaussian kernel density estimator, and a ...
0
votes
0answers
34 views

Derivation of AMISE and Bandwidth

Given: Let $K(\cdot)$ be a bona fide kernel. Let $f$ be a pdf and $\widehat{f}_n$ is kernel density estimator with bandwidth $h$ based on a sample $X_1,X_2,\cdots,X_n$ of size $n$ draw iid from $f$. ...
2
votes
0answers
13 views

Contribution of a predictor in Nonparametric regression

Is there an equivalent to a beta weights in a nonparametric regression? I am using the NP package in R and running a local linear regression where my bandwidth estimates are produced using least ...
3
votes
1answer
55 views

Properties of Kernel Density Estimators

Given Let $X \in \mathbb{R}$ be a real-valued random variable with theoretical probability density function (pdf) $f(x)$ and corresponding cumulative distribution function (cdf) $F(x)$. Let $X_1, X_2,...
1
vote
0answers
36 views

GARCH model with t-innovations

I am modelling a time series with GARCH model with t-distributed error using RUGARCH package. My model is specified as: ...
0
votes
0answers
27 views

Adaptive Parzen Estimator

I am trying to implement the Tree Parzen Estimator for hyperparameter optimization. I follow this paper https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf, in which ...
0
votes
0answers
48 views

Confidence Region of a multivariate KDE in Python?

I have an estimated bivariate kernel density based on a set of observations (𝑥11,𝑥12,...,𝑥1𝑛) and (𝑥21,𝑥22,...,𝑥2𝑛) and would like to draw confidence regions in the (𝑥1,𝑥2) space. This is in ...
0
votes
0answers
7 views

Comparison of Kernel Home-Range Utilisation Distribution between samples with unequal numbers of locations

I have a dataset containing GPS coordinates for a sample of animals in the same location at the same time (think of an aerial snapshot of a cows in a field), at two different dates. I have estimated ...
1
vote
0answers
38 views

How to perform MCMC integration when no prior over the integrated function is available? [closed]

As far as I can tell, MCMC integration (e.g. VEGAS) is performed by sampling from a distribution proportional to $f(x)$ using MCMC, then building a density estimator $g(x)$ using these samples (for ...
0
votes
0answers
11 views

Mairhuber-Curtis Theorem, Adaptive Bases and Neural Networks

I am reading the paper Positive Definite Kernels: Past, Present and Future by GE Fasshauer in which he describes computing with PD kernels. In section 3.2 he mentions the Mairhuber-Curtis Theorem, ...
1
vote
0answers
36 views

Is There an Optimal Bandwidth for Kernel Density Estimation at One Point

I am trying to estimate the density of a 2D distribution using KDE, but I only care about the accuracy right at the origin. I have read about methods to estimate the optimal bandwidth matrix when you ...
1
vote
0answers
68 views

Kernel density estimation (KDE) for data points with different variance

Consider the following situation: An experiment was done in 15 different conditions, and a value of the parameter 'A' was measured in each experiment (A can have any integer value between 0 and ...
0
votes
0answers
34 views

Data Normalization for Time Series Forecasting using Nonparametric Model

What are the specialities of applying locally weighted learning for time series forecasting? I am trying to apply a nonparametric model ($K$-NN Regression) to forecast daily load curve (entire time ...
0
votes
0answers
17 views

How to decide the weight of the locally logistic regression?

I have a problem of how to decide the weight of the logistic regression. My model is as below. For the general logistic regression, we have the likelihood of the model: $L(\theta)=\prod_{i=1}^{m}P ( ...
1
vote
2answers
166 views

How to compute Integrated Squared Error for kernel density estimation in R

I am working on R for kernel density estimation. I am testing different kernels and I need to evaluate them. I use next code for density estimation: ...
1
vote
1answer
61 views

downsampling a kde / combining kde and histogram

I'm calculating a KDE of one parameter (y, particle density) in bins of another parameter (x, distance from the origin). At ...
0
votes
1answer
19 views

Function dependent to nth order on a sample

From this paper: ... every multivariate density estimator that is in any reasonable sense nonparametric may be written in the form: $$ \hat{f}(y) = \frac{1}{n}\sum_{i=1}^n K_n(x_i, y) $$ ...
1
vote
0answers
23 views

Hypothesis testing in non-parametric regression

Say I have two processes/time series, $X = (X_{t_{1}},X_{t_{2}},\dots , X_{t_{n}})$ and $Y = (Y_{t_{1}},Y_{t_{2}},\dots , Y_{t_{n}})$ observed at times $t_i$ for $i=1,2,\dots, n$ where $0 < t_1 <...
0
votes
0answers
30 views

Equivalent Kernel - Bishop Chapter 3

I've been struggling to understand the Equivalent Kernel in Bishop's Pattern Recognition and Machine Learning book. Can somebody explain the following Figure (3.10) from chapter 3.3.3? (image taken ...
0
votes
0answers
30 views

Bilinear Process

how to generate a functional bilinear process such as: $ X_{n+1}= \int\psi(t,s)X_{n}(s)ds + \iint\phi(t,s,u)X_{n}(s)\varepsilon_{n}(u)dsdu + \varepsilon_{n+1}(t) $ ? About the first integral there ...
0
votes
0answers
48 views

Are vanishing bias and variance enough for pointwise consistency for KDE-based estimation?

Question: Is the condition that asymptotic bias and asymptotic variance goes to zero for infinite samples sufficient to guarantee the pointwise consistency of an estimator based on plug-in kernel ...
8
votes
2answers
798 views

What did Silverman (1981) mean by 'critical bandwidth'?

In the selection of a bandwidth for a Kernel Density Estimator, critical bandwidth according to my understanding is: "For every integer k, where 1<k<n, we ...
1
vote
0answers
55 views

Can kernel density estimation be used to estimate a radial probability density?

I am trying to estimate the PDF of the radius of points distributed in a 2D plane. The points are distributed like this: I can produce a histogram of the radius data which looks like this: I want ...
0
votes
0answers
31 views

Derive the estimator for the integrated squared bias $\int \left(\operatorname{E}\hat{f} - f\right)^2 $

This problem is found in p. 77 of Wand & Jones' (1995) book. If you are familiar with nonparametric estimation you may skip this introduction. Suppose we want to minimize the integrated squared ...
1
vote
0answers
42 views

Gaussian random fields: matrix and convolution sampling

I should be able to generate a stationary GRF from white noise in two different ways: multiplying the white noise vector by the square root of a covariance matrix with appropriate kernel; taking the ...
2
votes
0answers
322 views

Point process - intensity function vs probability density function

Suppose we have a point process in $\mathbb{R}$ with intensity $\lambda(x)$. Then, for a given compact set ${ S}$ we have $$\Lambda({ S})=\int_{\rm S} \lambda(x) \, dx,$$ where $\Lambda({ S})$ is ...
1
vote
1answer
41 views

How can I derive the function curve from a histogram of observed data

I'm analysing some datasets that produce heavy tailed data when plotted as a histogram. My initial goal was to attempt to fit a known distribution to my dataset. Thereafter I use to the properties of ...
0
votes
0answers
20 views

Is a kernel density estimate meaningful if > 25% of my data are duplicates?

The title pretty much says it all. I have data that consists of 80 samples but there are always at least four samples that have exactly the same value. I want to assess, whether the data is unimodal. ...
0
votes
0answers
12 views

What are the pitfals of using kernel density estimates to infer about the shape of an underlying distribution

I know that we cannot simply infer from the shape of a histogram to the shape of the underlying distribution, as the shape of the histogram is influenced by the choice of the intervals (Assessing ...
0
votes
1answer
48 views

Uniform kernel estimator

For practice, I'm trying to provide an estimation for a nonparametric model on dataset BMACS from library (npmlda). I'm having trouble to set up a kernel estimator with Uniform(-1/2,1/2) kernel and ...
2
votes
0answers
51 views

why not chosing always a spline?

I'm having a quite simple question: Why is a spline fit not the best choice everytime? In other words: How do I separate a spline fit from a kernel smoother or a polynomial in a meaningful way? I'm ...
5
votes
2answers
56 views

How to compare a new measurement to an existing multivariate distribution?

I have a dataset that describes the position and rotation of an object at different points in time using four dimensions. I want to use this sample of observations to get a sense of what positions and ...
1
vote
0answers
54 views

RKHS norm and Fourier transform link

In the notes here, it is stated that norms of some reproducing kernel Hilbert spaces can be written in terms of Fourier transforms, and this is often used to argue that a higher RKHS norm implies a ...
3
votes
0answers
68 views

Gradient Descent in Metric Learning for Kernel Regression (MLKR)

I am currently studying the Metric Learning for Kernel Regression (MLKR) algorithm (http://proceedings.mlr.press/v2/weinberger07a/weinberger07a.pdf). Let $\{(x_{1}, y_{1}), ..., (x_{N}, y_{N})\}$ ...
1
vote
0answers
13 views

Cluster/partition by time

I have a dataset of different events in time. I want to group/cluster/partition the data by datetime. A small example of the time of the data might be: [19-09-2018 12:00, 19-09-2018 12:01, 19-09-2018 ...
0
votes
1answer
168 views

MSE of Kernel Density Estimator

Erlang Kernel is used for density estimation. By using this estimates are pretty close to the real density on graph on the other side MSE is very large. But Author of Erlang Kernel stated that it will ...
0
votes
1answer
91 views

Bandwidth Selection Methods

Are there more methods for calculating the bandwidth in a kernel regression? So far I found Cross Validation Methods and a Bayesian approach (by Zhang, Brooks and King).