Questions tagged [kernel-smoothing]

Kernel smoothing techniques, such as kernel density estimation (KDE) and Nadaraya-Watson kernel regression, estimate functions by local interpolation from data points. Not to be confused with [kernel-trick], for the kernels used e.g. in SVMs.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Doubt about Bhattacharyya Coefficient

I wrote this code to compute the bhattacharyya coefficient between two probabilities distributions. The idea behind was to compute a kernel transformation on the original data, compute 3 estimators (...
0
votes
0answers
11 views

integrate multi-variate KDE over intervals to approximate probabilities in scipy

Background is I would like to work out the probabilities of certain events occurring to do this, Say I have three intervals: First (-inf, -1) Second ...
0
votes
0answers
14 views

How to interpret this kernel regression results?

I tried to do some kernel estimate of second order derivative. In my experiment, $y_i=z_{1i}+z_{1i}z_{2i}^3+e_i$, where $E(y_i|z_{1i},z_{2i})=z_{1i}+z_{1i}z_{2i}^3$. I'm interested in estimating $\...
1
vote
0answers
11 views

Is there new easy to use methods for bidimensional density estimation?

Recently, I was learning about density estimation with non-parametric methods, brilliantly detailed here by Jake VanderPlas. I'm just a beginner in the area, and I also heard about mixture models. I ...
9
votes
1answer
198 views

If I know the density I'm estimating is symmetric about 0, how to impose this restriction in my kernel density estimator?

Suppose I'm interested in estimating the unknown smooth density of $X$ denoted by $f(\cdot)$ using data $\{X_i\}_{i=1}^{n}$. Suppose I also know that $f(\cdot)$ is symmetric about 0 in the sense that $...
0
votes
1answer
21 views

Is it necessary to normalize the dataset before kernel density estimation?

Is it necessary to normalize (Z-score) the dataset (high dimension) when the dimensionality of features varies greatly? If I normalize the dataset, then the probability density (f1) obtained by KDE ...
1
vote
0answers
22 views

Which second order kernel is symmetric, has bounded support and satisfy $\int x^2 k(x)dx=1$?

Suppose $k(\cdot)$ is a univariate kernel function of order 2 in the sense that $\int x k(x)dx=0$, and $\int x^2 k(x)dx\neq0$. $k(\cdot)$ equals 0 outside a bounded interval, and $k(-x)=k(x)$ for any $...
1
vote
1answer
49 views

How does kernel density estimation work?

There is an inbuilt "adaptive Kernel density" anomaly detection routine provided in a data streaming library (https://docs.microsoft.com/en-us/stream-analytics-query/anomalydetection-...
0
votes
1answer
20 views

How determine the bandwidth of a gaussian kernel such that k nearest points represent a certain % of sum weight

I have a dataset to which I am applying a gaussian kernel. I want to determine the bandwidth (sd) of the kernel such that, on average, the k nearest points will represent a specified proportion of the ...
4
votes
1answer
100 views

Non-parametric (smoothed) estimate of current rate

I am looking at time-series event data and need to visualize how the arrival rate $\lambda$ changes over time. I do not want to assume any underlying distribution that the data comes from (it is ...
0
votes
0answers
12 views

Smoothing methods for unevenly sampled data

I have categorical, time-series data distributed in space. It is very noisy, but over the whole series there are big shifts in distribution - my goal is to see how these shifts progressed in space. So,...
0
votes
0answers
15 views

What is Gaussian kernel function in kernel density estimation of scipy.stats.gaussian_kde based on Scott rule?

I wanna know what is the Gauss kernel function in scipy.stats.gaussian_kde. According to source, we know scotts_factor=n**(-1./(d+4)), so what is the Gaussian kernel function for kernel density ...
0
votes
0answers
15 views

How to estimate the right derivative of a function that is smooth except for a finite number of kinks?

Suppose $Y_i=g(X_i)+e_i$, where $g(\cdot)$ is a function unknown to the researcher, and $E(e_i|X_i)=0$. Suppose $X_i$ is a random variable in $[-1,2]$ with a density that is everywhere positive, and ...
1
vote
1answer
20 views

What happens to kernel regression (Nadaraya–Watson estimator) at a kink point?

Suppose $Y_i=g(X_i)+e_i$, where $g(\cdot)$ is a function unknown to the researcher, and $E(e_i|X_i)=0$. Suppose $X_i$ is a random variable in $[-1,1]$ with a density that is everywhere positive, and ...
0
votes
0answers
11 views

Kernel density estimation of AR(1)

I recently started playing around with estimation joint densities with Matlab using its mvksdensity function and at the moment I am fooling around with an autoregressive process of order 1 (AR(1)) ...
2
votes
2answers
111 views

How to detect this change-point?

Let $Y\in\{0,1\}$ be a binary random variable. Let $P(x_1,x_2)=E(Y|x_1,x_2)$. By definition we have $0 \leq P(x_1,x_2) \leq 1$. Suppose $P(x_1,x_2)$ is strictly monotone in $x_1,x_2$ and $P(x_1,x_2)=1$...
0
votes
1answer
26 views

What is the order of this kernel function?

Suppose we have kernel function $K\left(t\right)=\frac{35}{32}\left(1-t^2\right)^3\mathbf{1}\left(|t|\leq 1\right)$. What is the minimal positive integer $r$ such that $\int K(t)t^rdt\neq0$?(this ...
1
vote
1answer
41 views

Kernel Density Uniform

Suppose that we have the vector $x = (11,10,5,14,10,10,16,9,13,10)$ we wish to adjust a kernel density $f$ to f where the Kernel(K) is a uniform(a,b) density. I understand that we can write $f(x) = \...
1
vote
1answer
39 views

Optimal Smoothing parameter for Uniform Kernel density function

Given an n-dimensional sample $\left\lbrace X_{i}\right\rbrace$ of i.i.d. observations, let's consider a kernel density function $$\hat{f}_{h_{n},n}(x)=\dfrac{1}{nh_{n}}\cdot\sum_{i=1}^n K\left(\dfrac{...
0
votes
0answers
21 views

How can I interpret a kernel density plot where x axis is the probability?

I am confused on how to interpret the kernel density estimation (kde) plot given below, which is of the predicted probabilities from my model for class 0 and class 1. What conclusions can I draw from ...
0
votes
0answers
16 views

Can I customize the kernel function?

I want to know whether I can customize the kernel function? For example, the polynomial kernel is defined as: $$ K(x,y) = (x^Ty+c)^d $$ Could I modify it to the following: $$ K(x,y) = (||x-y||_2)^d $$ ...
1
vote
1answer
20 views

kernel density estimation for more than one attribute

Consider the Iris dataset, which has 4 attributes. KDE(kernel density estimation) is univariate analysis then how can we get overall kernel density estimation of the iris dataset
1
vote
1answer
40 views

Why is this true?

suppose $T$ is a binary variable and $x$ is a continuous scalar, and $g(x)=E[T|x]$ is the conditional expectation of $T$. Suppose I estimate $g(x)$ using kernel regression $\widehat{g}(x)=\frac{\sum_{...
0
votes
0answers
47 views

Density estimation with High dimensional Data

I am a newbie learning about Kernel density estimations How density can be estimated for data with the number of samples equal to the number of features. Is it really useful to do density estimation ...
0
votes
0answers
7 views

Recovering Marginals from Multivariate Nadaraya-Watson

I am using Nadaraya–Watson kernel regression with multiple covariates as outlined here. I would like to estimate the effect of each covariate in the regression, and recovering the marginals seems like ...
0
votes
0answers
40 views

Derive an expression for the decision rule for a binary classification classifier

I want to derive the decision rule for the local constant logistic regression: Consider the log-likelihood for the GLM (general linearised model) \begin{equation} l( \beta_{0}, \beta_{1})= \sum_{i=1}^{...
0
votes
0answers
12 views

How to choose the number of samples to sample from an unbalanced dataset?

I have two unbalanced dataset with binary classes. One dataset has $13000$ samples for class 1 and $14000$ samples for class 2. Another dataset has $20000$ samples for class 1 and $40000$ samples for ...
2
votes
1answer
50 views

Kernel Bandwidth: Why scott's rule only use n**(-1./(d+4)) in scipy.stats

I have a question about bandwidth selection of kernel density estimate in scipy.stats. In the method, if we use Scott's rule, the bandwidth is equal to n**(-1./(d+4)), which means that the bandwidth ...
0
votes
0answers
17 views

Kernel density estimate of uniform pdf at the boundary

My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. How to show that for a uniform pdf between 0 and 1 with symmetric kernel K, when ...
0
votes
0answers
22 views

What's the relationship between standard deviation and bandwidth in a Gaussian kernel smoother? (ksmooth in R vs gaussian_filter in Python) [duplicate]

I am applying a Gaussian filter to smooth my data in Python, specifically I am applying the scipy function gaussian_filter1d. ...
0
votes
0answers
15 views

How to sample biased distribution from another distribution?

I have a PSNR histogram distribution for 6k images and I want to sample a few images from this distribution. But as you can see the distribution is right-skewed and the samples will less PSNR values ...
0
votes
0answers
14 views

evaluating an empirical multivariate PDF in python

I have multivariate (bivariate in the simplest case) residuals from a VAR time series regression and I'd like to estimate the joint pdf and then be able to draw from this pdf. If I have bivariate ...
0
votes
0answers
14 views

Distance between two multivariate kernel density estimates

I tried using the ks2samp to compare two multivariate kernel density estimates. Is it the right way to compare two multivariate densities?
0
votes
1answer
48 views

Derivation of variance for kernel density estimator

My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. Here, the variance for a kernel density estimator using the pointwise perspective (for fixed x) is ...
0
votes
0answers
45 views

Performing residual bootstrap using kernel regression in R

Kernel regression is a non-parametric technique that wants to estimate the conditional expectation of a random variable. It uses local averaging of the response value, Y, in order to find some non-...
0
votes
0answers
23 views

Kernel density estimates comparison in a multivariate setting

I have a dataset with 64 features and binary labels (class 1 and class 2). Before I fit any classification models, I wanted to check whether the samples belonging to the two classes come from the same ...
0
votes
0answers
31 views

Interpreting exercise in Elements of Statistical Learning

I am reading exercise 6.4 from The Elements of Statistical Learning (Hastie, Tibshirani and Friedman) and I am having difficulty interpreting exactly what is being asked in the following question ...
0
votes
0answers
6 views

Are there R packages with other kernel smoothers besides normal and box?

I was playing with kernel smoothing time series, and noticed that ksmooth only allows Gaussian and box kernels. Are there any other packages that do kernel smoothing with other kernels?
1
vote
1answer
29 views

Generating Irrelevant Variables for a Kernel Regression

Assume that my data generating process is $$y = \frac{1}{1 + \exp(-x_1)} + u,$$ where $u$ is normally distributed with mean $0$ and small variance. I sample from this process by randomly sampling 100 ...
0
votes
0answers
11 views

Efficiently comparing bivariate kernel density estimates

I have some bivariate kernel density estimates that I want to compare visually across different regions in space. The main features that I want to highlight are the shapes of the distributions and ...
2
votes
3answers
94 views

How to avoid smoothing to 0 at edges of R density plot

When using the density function in R, it includes smooth transitions down to 0 at both ends of the data. Is there a way to prevent this? As a trivial example, ...
0
votes
1answer
23 views

Fit cdf and pdf from points on empirical cdf

I have cumulative counts with respect to a variable x, which looks like: ...
0
votes
0answers
22 views

Non asymptotic error bound for non parametric estamation $f(x)=\mathbb{E}[Y|X=x]$

I am considering the following model: $(X_i,Y_i)_{i=1}^n$ are iid random pairs with $X_i\in[0,1]$ and $Y_i\in\mathbf{R}$. Let $f(x)=\mathbb{E}[Y|X=x]$. Consider an estimate $\hat{f}_n$ of $f$. For ...
0
votes
0answers
27 views

Estimate a Mean using Monte Carlo Integration

Suppose $\hat f(x)$ is the KDE $$\hat f(x) = \frac{1}{nh}\sum_{i=1}^nK\left(\frac{x - X_i}{h}\right).$$ Now I want to estimate the KL divergence to the true density $f$ using an MC approach: $$KL = \...
0
votes
0answers
22 views

Create KDE based on Histogram using the weighted KDE approach

In Kernel density estimation with binned data a binned kernel estimator is defined as : $g_n(x) = \frac{1}{nh} \sum_{j=-\infty}^{\infty} n_j K(\frac{x-t_j}{h})$ where $t_j$ denotes the centre of the ...
0
votes
0answers
7 views

k-nearest neighbor kernel density estimation for a an unknown stochastic process?

Suppose we have a sequence of random variables $\{X_t:t=1,\cdots,T\}$ following an unknown stochastic process (possibly stationary or non-stationary). Now I have two questions: 1- Would it be ...
0
votes
0answers
38 views

Prediction error of interpolation using Gaussian kernel smoothed intensity

I'll like to know how to calculate the prediction error (or interpolation error) between Gaussian kernel smoothed intensity images created if I comparing all data set (census) with a sample of the ...
2
votes
0answers
18 views

Examples for integration estimator

suppose I'm interested in estimating $C=\int_{a}^{b}g(x)dx$, where $a$ and $b$ are known, and $g(x)=E(Y|X=x)$ is an unknown function of $x$. The data I have is $\{Y_{i},X_{i}\}_{i=1}^{n}$, then a ...
0
votes
0answers
19 views

asymptotic properties for a local linear estimator evaluated at the boundary vs interior

Suppose I have local linear estimator at $x$ defined as $(\widehat{a}_{x},\widehat{b}_{x})=\underset{a_{x},b_{x}}{argmin}\sum_{i=1}^{n}(Y_{i}-a_{x}-b_{x}(X_{i}-x))^2k(\frac{X_{i}-x}{h})$, where $x$ ...
0
votes
0answers
9 views

How to assign weights using Kernel function based on a vector of pairwise Euclidean distance?

I want to quantify the dissimilarity between two group. Each group has 5 observations, so there are 25 combinations. For each combination, I have calculated their pairwise Euclidean distance (based ...

1
2 3 4 5
11