Questions tagged [nonparametric-regression]

Nonparametric regression is a form of regression analysis where the form of the functional dependence of the response on the predictors is not assumed. It subsumes many kinds of models, like spline models, kernel regression, gaussian process regression, regression trees or random forrests, and others.

Filter by
Sorted by
Tagged with
2 votes
1 answer
50 views

How well do Multivariate Adaptive Regression Splines work in high dimensional settings?

I have been reading the Hastie and Tibshirani book again lately, and I noticed in Chapter 9 that the mention the MARS algorithm: Multivariate Adaptive Regression Splines, which is a nonparametric ...
user avatar
  • 1,118
3 votes
1 answer
83 views

Calculating local variance

I have some data, and I assume it can be modelled by $y_i = f(x_i) + \epsilon $, where $\epsilon \sim \mathcal{N}(0,\sigma_0^2)$ where $f$ and $\sigma_0^2$ are unknown. I understand that I can ...
user avatar
1 vote
0 answers
19 views

Method of Sieves with Data Driven Basis Functions

Consider a nonparametric regression problem with i.i.d. sampled data $(y_1,x_1), (y_2,x_2),\ldots, (y_n,x_n)$ and regression function $$y_i = g_0(x_i) + \varepsilon_i,\quad \mathbb E[\varepsilon_i | ...
user avatar
  • 1,461
0 votes
1 answer
20 views

Rates of convergence with asymptotically negligibly noisy observations

Apologies in advance if this question is not completely well defined. Suppose that I am estimating a nonparametric model for a conditional expectation function $\mathbb E[Y_i | X_i]$ using some i.i.d. ...
user avatar
  • 1,461
3 votes
1 answer
68 views

Why isn't every nonparametric model with random model design an additive noise model?

Let $Y$ be a real random variable and $X$ be a real random vector. In a nonparametric model with additive noise, we assume the relationship $$Y = f(X) + \epsilon$$ for some unknown regression function ...
user avatar
1 vote
1 answer
126 views

R: "family" and "degree" specification in loess fitting

I can't understand the difference between the possible specifications of the family option in the loess command in R. This is ...
user avatar
0 votes
0 answers
158 views

why is the nadaraya watson estimator unbiased?

Say I have the model $Y_{i} = m(x_{i}) + \epsilon_{i}$ and $Y_{i}$ and $X_{i}$ are two mutually independent i.i.d. sequences. Then, how can I show that the Nadaraya Watson estimator is unbiased for ...
user avatar
0 votes
1 answer
22 views

Is the distinction between parametric and non-parametric statistics always clear-cut?

Is the distinction between parametric and non-parametric statistics always clear-cut or do examples of a statistic exists which cannot clearly assigned to one of these categories?
user avatar
  • 2,502
0 votes
0 answers
90 views

can I estimate density function of 2d/3d data with kernel smoothing (e.g. ks package R), or are there better estimation methods

I have a 2d matrix of positive values (non integer), where the values can be thought of intensity at an x,y coordinate indexed by the row and column. I want to estimate a density function across this ...
user avatar
  • 1
0 votes
1 answer
32 views

The nonparametric estimation in generalized regression model

Let $Y_t \in \mathbb{R}$ be a response variable and $X_t$ a $d$-dimensional explanatory variable. Assume we observe the process that $(X_1, Y_1), \cdots, (X_n, Y_n)$. \begin{equation} Y_{t} = \mu(...
user avatar
0 votes
1 answer
774 views

Gaussian and Epanechnikov Kernel Regressions giving drastically different estimations

sorry if this is the wrong place to be asking this question. I'm trying to implement kernel regression for a specific dataset I'm working with, but I'm noticing that the trendlines generated by my ...
user avatar
0 votes
1 answer
52 views

Nonparametric Regression

Suppose I have response y, continuous independent variable x and binary variable z. ...
user avatar
0 votes
1 answer
49 views

Can I use/make prediction/regression if my data is not normally distributes? Are non-parametric test for prediction?

My data is not normally distributed, and I`m confused what tests can I use (non-parametric, of course), but is there any way, to analyse prediction if the data is not normally distributed? I read ...
user avatar
1 vote
0 answers
12 views

About Generalized Additive Models - First parametric estimations, after nonparametric estimations for the remaining components

I wonder is it possible to construct a generalized linear modelin in that way, First, I will exclude the intercept term, which is standard for GAMs. Second, I will predict my interested dependent ...
user avatar
  • 639
3 votes
1 answer
43 views

A particular method for estimating the gradient of a log-density from samples

Suppose I have $N$ samples $x^1, \ldots, x^N$ which were drawn iid from an unknown density $P(x)$. Suppose I am interested in estimating the vector-valued function $g(x) = \nabla \log P (x)$. One ...
user avatar
  • 1,286
2 votes
0 answers
22 views

Examples for integration estimator

suppose I'm interested in estimating $C=\int_{a}^{b}g(x)dx$, where $a$ and $b$ are known, and $g(x)=E(Y|X=x)$ is an unknown function of $x$. The data I have is $\{Y_{i},X_{i}\}_{i=1}^{n}$, then a ...
user avatar
  • 1,682
1 vote
0 answers
17 views

rate of convergence for cross derivative estimation in local linear regression

Suppose $Y_{i}=m(X_{1i},X_{2i})+\epsilon_{i}$, with $E(Y_{i}|X_{1i},X_{2i})=m(X_{1i},X_{2i})$ where $m(\cdot,\cdot)$ is an unknown smooth function. If the estimator $\widehat{m}(x_{1},x_{2})$ is ...
user avatar
  • 1,682
4 votes
0 answers
93 views

Intuition of the regression model under fixed design case (nonparametric regression)

Let $(x_1,Y_1), \dotsc, (x_n,Y_n)$ be a random sample from the regression model $$Y_t=m(x_t)+\epsilon_t.$$ When authors want to develop the asymptotic properties of the local linear estimator of $m$ ...
user avatar
3 votes
1 answer
92 views

Rates of convergence for estimating population mean squared error

Suppose I have an i.i.d. sample $\{(Y_i, X_i)\}_{i=1}^n$ on which I am trying to estimate a conditional expectation model: $$Y = g(X) + \varepsilon,\quad \mathbb E[\varepsilon | X] = 0$$ There is a ...
user avatar
  • 1,461
0 votes
1 answer
326 views

How robust is coxph when the proportional hazards assumption is violated?

How robust is the coxph when I don’t have proportional hazards? How common is non prop hazards and how do I fix it? Does transforming variables help? Does non parametric survival analysis handle non ...
user avatar
2 votes
0 answers
98 views

How Semiparametric regression works?

I am working on semiparametric regression models; $$y=\beta x_1 +m(x_2)+e$$. I can understand this combination of Parametric and Nonparametric but how to estimate the responses ($\hat y$)? What is ...
user avatar
3 votes
0 answers
98 views

Multiple regression for left-censored independent and dependent variables

I am interested in developing a predictive multiple regression model which predicts a concentration of one compound based on the measured concentrations of several other compounds. Both the dependent ...
user avatar
1 vote
1 answer
217 views

For non-parametric regression which one has better interpretation and properties, GAM or quantile regression?

As in the topic. I want to interpret data for which I have no clues about the distribution. It's neither count, percentage, continuous. I don't want any transformations. Instead I would like to ...
user avatar
2 votes
0 answers
27 views

Rate of convergence of variance of kernel averages

I'm reading Hansen's (2008, p. 729) Theorem 1 where he bounds the variance of averages of the form $$\hat\Psi(x)=\frac{1}{Th}\sum_{t=1}^T Y_t K\bigg(\frac{x-X_t}{h}\bigg)$$ given that $\{(Y_t,X_t)\}_{...
user avatar
1 vote
0 answers
64 views

Is there a nonparametric regression to analyze repeated measures

I get involved in a study with only 40 subjects, and each subject has repeated measures at 6 months, 12 months, and 24 months. Originally I supposed that the total sample size 40x3 = 120 should be ...
user avatar
  • 368
2 votes
0 answers
81 views

In Gaussian Process regression is there a way to force the prior slope to be positive when using a linear kernel?

I'm doing Gaussian process regression on some data $X$ with low sample size, using a squared exponential kernel. From domain knowledge I know that outside the range of my data the regressed function ...
user avatar
  • 71
0 votes
0 answers
27 views

How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?

I have 2 input variables, $X_1$ and $X_2$ that affect output variable $Y$. I can run experiments where I modify the inputs and measure what happens to the output. Now, if $X_1$ and $X_2$ were binary, ...
user avatar
  • 1,228
2 votes
0 answers
112 views

A comparison of the global optimal binwidth and local optimal binwidth of the histogram estimator

Suppose we have $X_1, \dots, X_n$ to be an i.i.d sample with unknown pdf $f(x)$ and cdf $F(x)$, and define $\hat{f} (x)$ to be the histogram estimator. We also define its Mean Integrated Square Error ...
user avatar
  • 183
1 vote
1 answer
65 views

Nonparametric approach for regression with a quadratic fit

I'm trying to figure out which nonparametric test I should run on my data. My data has residuals that are not normal, so I cannot run a linear regression unless I log transform it. However, log ...
user avatar
  • 11
1 vote
0 answers
220 views

Using Mean Cumulative Function (Nelson-Aalen) to assess drug efficacy

I have a large medical retrospective longitudinal dataset of electronic health records. An individual is identified by an ID. A medical event or drug prescription event is identified using a code and ...
user avatar
2 votes
3 answers
6k views

What is the cost/loss function of K nearest neighbors?

I am able to visualize how KNN works. Essentially take avg of the k nearest train neighbors for regression problem. However every ML algorithm optimizes a cost/loss function for example: Linear ...
user avatar
1 vote
0 answers
39 views

Nonparametric regression on data with known noise parameterization

What's the best way to regress on data for which we don't have a parameterised generative model (e.g. an arbitrary non-smooth continuous signal, that can be regressed in model-free ways with splines, ...
user avatar
  • 884
3 votes
0 answers
50 views

Nonparametric regression with missing data

Let $(y_i,x_i,b_i)$ be data at hand, where $y_i$ is a response variable, $x_i$ is covariates, and $b_i$ is an indicator for missing: if 1, then $y_i$ is observable, 0 otherwise. Then, under missing at ...
user avatar
1 vote
0 answers
95 views

Finding the overlap of two Bivariate Non Parametric models using R

I need a few suggestions on some methods to try or to be pointed in the right direction. I will start off describing the big picture. I have little stats background. I come from physics and astronomy. ...
user avatar
2 votes
1 answer
4k views

Time complexity of leave-one-out cross validation for nonparametric regression?

From Artificial Intelligence: A modern approach: Most nonparametric models have the advantage that it is easy to do leave-one-out crossvalidation without having to recompute everything. With a k-...
user avatar
3 votes
1 answer
76 views

What is the density of $X$ under fixed design?

We observe an i.i.d. sample $(X_1, Y_1), \ldots (X_n, Y_n).$ Let $m(x) = E(Y|X=x),$ $\sigma^2(x) = \operatorname{Var}(Y|X=x)$ and let $f(\cdot)$ be the density of $X.$ Under some regularity ...
user avatar
  • 218
1 vote
0 answers
511 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
user avatar
  • 218
5 votes
0 answers
517 views

Is calculating a moving average a good way to approximate k-nearest neighbor regression?

Given i.i.d samples (x1,y1), ... (xn,yn) such that yi = f0(xi) + $\epsilon$i, i = 1,... n for some f0 Suppose I want an estimate $\hat{f}$ of f0 using k-nearest-neighbors regression in the ...
user avatar
0 votes
0 answers
182 views

Examples of using MCMC for GP regression

This is a reference request. I am in a position of needing to use MCMC to do Gaussian process regression for a project. I have used MCMC before, and I have used GPs before, but never together. It ...
user avatar
  • 278
1 vote
0 answers
188 views

What is the basic difference between Sieves, Series and Splines estimators?

As far as I know, Sieve Estimators consists in a broader class of estimators for a function g(x) lying in a space of functions G. The estimation basically consists in choosing the function that best ...
user avatar
1 vote
0 answers
233 views

What is the nonparametric equivalent of the Weighted-Least-Squares-Regression?

I am struggling to find the nonparametric equivalent of the WLSR. In brief, I need a nonparametric regression method which allows to assign different weights to data according to the uncertainty. I ...
user avatar
  • 163
0 votes
0 answers
208 views

approximate a nonparametric CDF in R

I have two vectors of same length. The first vector is a collection of realization from an unknown random variable. The second vector is the distribution computed at each particular realization. A ...
user avatar
  • 61
0 votes
1 answer
197 views

Skewed Response Variable a lot zeroes

I am trying to build a non-parametric model to predict pure premium of an insurance policy. Pure premium is simply the expected claim amount of an insurance policy.The problem is, most insurance ...
user avatar
2 votes
0 answers
155 views

Is it appropriate to apply ordinal regression to a continuous response "as is" (with no ties)?

The goal is to run non-parametric general linear model with no restrictions on the design (e.g. not limited to one-way ANOVA as in Kruskal-Wallis test; quantitative factors should be admissible, etc) ...
user avatar
  • 2,650
2 votes
1 answer
974 views

How to choose between parametric and non-parametric regressions? [closed]

I am new to nonparametric regressions. What tests should one perform to choose a non-parametric regression model over a parametric one(Or vice versa)? Let's assume in our analysis we have a continuous ...
user avatar
3 votes
0 answers
417 views

Options for non-linear or non-parametric count data regression in R?

I am trying to move from a current parametric Poisson regression to a non-parametric count regression and would appreciate views on the best way to do this. Current state of analysis I have a time ...
user avatar
  • 151
0 votes
1 answer
475 views

How to fit nonparametric mixtures of regression models in a statistical program? [closed]

I am interested in estimating a nonparametric finite mixture regression model, explained for example here: https://methodology.psu.edu/media/techreports/09-93.pdf But I don't know with which program ...
user avatar
3 votes
1 answer
102 views

Why instrumental variables? Or: why not nonparametric regression?

Usually instrumental variables are introduced as a means to solve the problem $E(u|X)\neq 0$ in the model $Y = X'\beta + u$. This may happen if we omit important variables from the covariate vector $X$...
user avatar
3 votes
0 answers
297 views

Which non-parametric multiple-regression methods are computationally efficient with respect to the number of regressors?

I did some regression in R with random forests and got some decent results, $1-\sum{|e_i|}/\sum{|y_i-\bar{y}|}=0.692$, but I want to do better than this. Through my research, I have concluded that the ...
user avatar
1 vote
1 answer
617 views

Reporting the results of nonparametric regression using kernel weights

I am wondering how I can present the results of nonparametric regression. I performed the nonparametric tests using R, and R package 'np'. The commands used for this are freq <- npreg(Respno ~ ...
user avatar
  • 101