Questions tagged [nonparametric-regression]
Nonparametric regression is a form of regression analysis where the form of the functional dependence of the response on the predictors is not assumed. It subsumes many kinds of models, like spline models, kernel regression, gaussian process regression, regression trees or random forrests, and others.
71
questions
2
votes
1
answer
47
views
How to test whether a nonparametric function is equal to 0?
There is an unknown function $h(x)=E(Y|X=x)$ which I estimated with a nonparametric series estimator (also called sieve estimator) $\widehat{h}(x)$ using data $\{Y_i,X_i\}_{i=1}^n$. I'm interested in ...
- 2,956
0
votes
0
answers
16
views
nonparametric model for longitudinal data analysis
For the longitudinal data provided below, we have the following variables: the response variable 'y', the time variable 'week', 'grp' (with two levels: treatment and control), and 'subject'. My ...
- 1,173
0
votes
0
answers
14
views
Estimation of bivariate function with one variable being constricted
Suppose the following classical supervised regression setting,
$$y_{i} = f(x_{i}) + \epsilon_{i}, \quad i=1,\cdots,n,$$
where $\epsilon_{i}$ are i.i.d. zero mean Gaussian noise.
The above regression ...
- 11
5
votes
1
answer
254
views
Convergence rate of a nonparametric estimator
Optimal rate of convergence for a nonparametric estimator is well-known. This rate is derived for when we don't anything about functional form (expect perhaps degree of smoothness). Suppose we know ...
- 531
0
votes
0
answers
10
views
Variable independece in marginal integration estimator
This is an exercise of the textbook Nonparametric and Semiparametric Models by Wolfgang Hardle
Exercise 8.1. Assume that the regressor variable $X_1$ is independent from $X_2, \cdots, X_d$. How does ...
- 111
0
votes
0
answers
18
views
References: convergence rates of kernel regression, exchangeable data
I have been studying Kernel estimation; in particular, the Nadaraya-Watson estimator. I am interested in studying the rate of convergence in L^p of the NW (or similar) estimators for subgaussian ...
0
votes
0
answers
13
views
Where can I find analysis on the convergence rate of RMSE for various algorithms?
I am looking for convergence rates of the RMSE of various machine learning algorithms and conditions for them. Essentially, I would like to find something of the form
$$E[ |\hat{f}(x) - f(x)|^2] = o(1/...
1
vote
0
answers
74
views
Gaussian Process Regression prior with observations as integrals?
Consider some standard 1d Gaussian Process Regression (GPR). Suppose you are not happy with a typical mean-zero prior away from the data and you actually want something like the derivative of the mean ...
- 312
1
vote
0
answers
40
views
Maximum bias for NW estimator when $r(x)$ is Lipschitz (question 17, chapter 5 All of Non-Parametric Statistics)
The general condition is that $Y_i = r(X_i) + \epsilon_i$, and we want to estimate $r$ using Nadaraya–Watson kernel regression.
We additionally assume $r\colon [0,1] \to \mathbb{R}$ is lipschitz, so $|...
- 656
1
vote
0
answers
108
views
Is Synthetic Control Method a nonparametric estimator?
I'm studying causal inference and I'm struggling to understand how to properly classify an estimator as nonparametric. My colleague argued that the Synthetic Control Method is an example of a ...
- 439
3
votes
0
answers
486
views
Pros and cons of Nadaraya–Watson estimator vs. RKHS method?
Recently I've been reading some materials about nonparametric methods. Two methods related to the word "kernel" rasied my interest-- Nadaraya–Watson estimator and RKHS method.
What's the ...
- 31
2
votes
1
answer
361
views
How well do Multivariate Adaptive Regression Splines work in high dimensional settings?
I have been reading the Hastie and Tibshirani book again lately, and I noticed in Chapter 9 that the mention the MARS algorithm: Multivariate Adaptive Regression Splines, which is a nonparametric ...
- 1,532
4
votes
1
answer
477
views
Calculating local variance
I have some data, and I assume it can be modelled by $y_i = f(x_i) + \epsilon $, where $\epsilon \sim \mathcal{N}(0,\sigma_0^2)$ where $f$ and $\sigma_0^2$ are unknown. I understand that I can ...
- 327
2
votes
1
answer
168
views
Method of Sieves with Data Driven Basis Functions
Consider a nonparametric regression problem with i.i.d. sampled data $(y_1,x_1), (y_2,x_2),\ldots, (y_n,x_n)$ and regression function
$$y_i = g_0(x_i) + \varepsilon_i,\quad \mathbb E[\varepsilon_i | ...
- 2,495
1
vote
1
answer
32
views
Rates of convergence with asymptotically negligibly noisy observations
Apologies in advance if this question is not completely well defined. Suppose that I am estimating a nonparametric model for a conditional expectation function $\mathbb E[Y_i | X_i]$ using some i.i.d. ...
- 2,495
3
votes
1
answer
112
views
Why isn't every nonparametric model with random model design an additive noise model?
Let $Y$ be a real random variable and $X$ be a real random vector. In a nonparametric model with additive noise, we assume the relationship
$$Y = f(X) + \epsilon$$
for some unknown regression function ...
- 41
1
vote
1
answer
610
views
R: "family" and "degree" specification in loess fitting
I can't understand the difference between the possible specifications of the family option in the loess command in R. This is ...
- 111
0
votes
0
answers
459
views
why is the nadaraya watson estimator unbiased?
Say I have the model $Y_{i} = m(x_{i}) + \epsilon_{i}$ and $Y_{i}$ and $X_{i}$ are two mutually independent i.i.d. sequences.
Then, how can I show that the Nadaraya Watson estimator is unbiased for ...
1
vote
1
answer
30
views
Is the distinction between parametric and non-parametric statistics always clear-cut?
Is the distinction between parametric and non-parametric statistics always clear-cut or do examples of a statistic exists which cannot clearly assigned to one of these categories?
- 3,092
0
votes
0
answers
165
views
can I estimate density function of 2d/3d data with kernel smoothing (e.g. ks package R), or are there better estimation methods
I have a 2d matrix of positive values (non integer), where the values can be thought of intensity at an x,y coordinate indexed by the row and column. I want to estimate a density function across this ...
- 1
1
vote
1
answer
51
views
The nonparametric estimation in generalized regression model
Let $Y_t \in \mathbb{R}$ be a response variable and $X_t$ a $d$-dimensional explanatory variable. Assume we observe the process that $(X_1, Y_1), \cdots, (X_n, Y_n)$.
\begin{equation}
Y_{t} = \mu(...
- 203
0
votes
1
answer
2k
views
Gaussian and Epanechnikov Kernel Regressions giving drastically different estimations
sorry if this is the wrong place to be asking this question.
I'm trying to implement kernel regression for a specific dataset I'm working with, but I'm noticing that the trendlines generated by my ...
- 15
0
votes
1
answer
116
views
Nonparametric Regression
Suppose I have response y, continuous independent variable x and binary variable z.
...
- 357
0
votes
1
answer
71
views
Can I use/make prediction/regression if my data is not normally distributes? Are non-parametric test for prediction?
My data is not normally distributed, and I`m confused what tests can I use (non-parametric, of course), but is there any way, to analyse prediction if the data is not normally distributed? I read ...
1
vote
0
answers
13
views
About Generalized Additive Models - First parametric estimations, after nonparametric estimations for the remaining components
I wonder is it possible to construct a generalized linear modelin in that way,
First, I will exclude the intercept term, which is standard for GAMs.
Second, I will predict my interested dependent ...
- 699
3
votes
1
answer
104
views
A particular method for estimating the gradient of a log-density from samples
Suppose I have $N$ samples $x^1, \ldots, x^N$ which were drawn iid from an unknown density $P(x)$.
Suppose I am interested in estimating the vector-valued function $g(x) = \nabla \log P (x)$.
One ...
- 1,356
2
votes
0
answers
36
views
Examples for integration estimator
suppose I'm interested in estimating $C=\int_{a}^{b}g(x)dx$, where $a$ and $b$ are known, and $g(x)=E(Y|X=x)$ is an unknown function of $x$. The data I have is $\{Y_{i},X_{i}\}_{i=1}^{n}$, then a ...
- 2,956
1
vote
0
answers
38
views
rate of convergence for cross derivative estimation in local linear regression
Suppose $Y_{i}=m(X_{1i},X_{2i})+\epsilon_{i}$, with $E(Y_{i}|X_{1i},X_{2i})=m(X_{1i},X_{2i})$ where $m(\cdot,\cdot)$ is an unknown smooth function. If the estimator $\widehat{m}(x_{1},x_{2})$ is ...
- 2,956
4
votes
0
answers
196
views
Intuition of the regression model under fixed design case (nonparametric regression)
Let $(x_1,Y_1), \dotsc, (x_n,Y_n)$ be a random sample from the regression model
$$Y_t=m(x_t)+\epsilon_t.$$
When authors want to develop the asymptotic properties of the local linear estimator of $m$ ...
- 1,097
3
votes
1
answer
423
views
Rates of convergence for estimating population mean squared error
Suppose I have an i.i.d. sample $\{(Y_i, X_i)\}_{i=1}^n$ on which I am trying to estimate a conditional expectation model:
$$Y = g(X) + \varepsilon,\quad \mathbb E[\varepsilon | X] = 0$$
There is a ...
- 2,495
0
votes
1
answer
705
views
How robust is coxph when the proportional hazards assumption is violated?
How robust is the coxph when I don’t have proportional hazards? How common is non prop hazards and how do I fix it? Does transforming variables help? Does non parametric survival analysis handle non ...
user271077
2
votes
0
answers
137
views
How Semiparametric regression works?
I am working on semiparametric regression models;
$$y=\beta x_1 +m(x_2)+e$$. I can understand this combination of Parametric and Nonparametric but how to estimate the responses ($\hat y$)?
What is ...
- 21
4
votes
0
answers
186
views
Multiple regression for left-censored independent and dependent variables
I am interested in developing a predictive multiple regression model which predicts a concentration of one compound based on the measured concentrations of several other compounds. Both the dependent ...
- 61
2
votes
1
answer
495
views
For non-parametric regression which one has better interpretation and properties, GAM or quantile regression?
As in the topic. I want to interpret data for which I have no clues about the distribution. It's neither count, percentage, continuous. I don't want any transformations. Instead I would like to ...
- 337
2
votes
0
answers
41
views
Rate of convergence of variance of kernel averages
I'm reading Hansen's (2008, p. 729) Theorem 1 where he bounds the variance of averages of the form
$$\hat\Psi(x)=\frac{1}{Th}\sum_{t=1}^T Y_t K\bigg(\frac{x-X_t}{h}\bigg)$$
given that $\{(Y_t,X_t)\}_{...
- 1,097
1
vote
0
answers
139
views
Is there a nonparametric regression to analyze repeated measures
I get involved in a study with only 40 subjects, and each subject has repeated measures at 6 months, 12 months, and 24 months. Originally I supposed that the total sample size 40x3 = 120 should be ...
- 498
2
votes
0
answers
152
views
In Gaussian Process regression is there a way to force the prior slope to be positive when using a linear kernel?
I'm doing Gaussian process regression on some data $X$ with low sample size, using a squared exponential kernel. From domain knowledge I know that outside the range of my data the regressed function ...
- 81
0
votes
0
answers
31
views
How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?
I have 2 input variables, $X_1$ and $X_2$ that affect output variable $Y$. I can run experiments where I modify the inputs and measure what happens to the output.
Now, if $X_1$ and $X_2$ were binary, ...
- 1,118
2
votes
0
answers
116
views
A comparison of the global optimal binwidth and local optimal binwidth of the histogram estimator
Suppose we have $X_1, \dots, X_n$ to be an i.i.d sample with unknown pdf $f(x)$ and cdf $F(x)$, and define $\hat{f} (x)$ to be the histogram estimator. We also define its Mean Integrated Square Error ...
- 193
1
vote
1
answer
111
views
Nonparametric approach for regression with a quadratic fit
I'm trying to figure out which nonparametric test I should run on my data.
My data has residuals that are not normal, so I cannot run a linear regression unless I log transform it. However, log ...
- 11
1
vote
0
answers
334
views
Using Mean Cumulative Function (Nelson-Aalen) to assess drug efficacy
I have a large medical retrospective longitudinal dataset of electronic health records. An individual is identified by an ID. A medical event or drug prescription event is identified using a code and ...
- 605
2
votes
3
answers
8k
views
What is the cost/loss function of K nearest neighbors?
I am able to visualize how KNN works. Essentially take avg of the k nearest train neighbors for regression problem.
However every ML algorithm optimizes a cost/loss function for example: Linear ...
- 5,653
1
vote
0
answers
40
views
Nonparametric regression on data with known noise parameterization
What's the best way to regress on data for which we don't have a parameterised generative model (e.g. an arbitrary non-smooth continuous signal, that can be regressed in model-free ways with splines, ...
- 964
3
votes
0
answers
60
views
Nonparametric regression with missing data
Let $(y_i,x_i,b_i)$ be data at hand, where $y_i$ is a response variable, $x_i$ is covariates, and $b_i$ is an indicator for missing: if 1, then $y_i$ is observable, 0 otherwise. Then, under missing at ...
- 657
1
vote
0
answers
122
views
Finding the overlap of two Bivariate Non Parametric models using R
I need a few suggestions on some methods to try or to be pointed in the right direction.
I will start off describing the big picture.
I have little stats background. I come from physics and astronomy. ...
2
votes
1
answer
4k
views
Time complexity of leave-one-out cross validation for nonparametric regression?
From Artificial Intelligence: A modern approach:
Most nonparametric models have the advantage that it is easy to do leave-one-out crossvalidation
without having to recompute everything. With a k-...
- 153
3
votes
1
answer
88
views
What is the density of $X$ under fixed design?
We observe an i.i.d. sample $(X_1, Y_1), \ldots (X_n, Y_n).$ Let $m(x) = E(Y|X=x),$ $\sigma^2(x) = \operatorname{Var}(Y|X=x)$ and let $f(\cdot)$ be the density of $X.$
Under some regularity ...
- 238
2
votes
0
answers
761
views
Understanding a Taylor expansion for the bias of local polynomial regression
I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$.
Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
- 238
6
votes
0
answers
633
views
Is calculating a moving average a good way to approximate k-nearest neighbor regression?
Given i.i.d samples (x1,y1), ... (xn,yn) such that yi = f0(xi) + $\epsilon$i, i = 1,... n for some f0
Suppose I want an estimate $\hat{f}$ of f0 using k-nearest-neighbors regression in the ...
- 779
1
vote
0
answers
254
views
Examples of using MCMC for GP regression
This is a reference request. I am in a position of needing to use MCMC to do Gaussian process regression for a project. I have used MCMC before, and I have used GPs before, but never together. It ...
- 318