Questions tagged [nonparametric-regression]

Nonparametric regression is a form of regression analysis where the form of the functional dependence of the response on the predictors is not assumed. It subsumes many kinds of models, like spline models, kernel regression, gaussian process regression, regression trees or random forrests, and others.

Filter by
Sorted by
Tagged with
3
votes
0answers
15 views

Multiple regression for left-censored independent and dependent variables

I am interested in developing a predictive multiple regression model which predicts a concentration of one compound based on the measured concentrations of several other compounds. Both the dependent ...
1
vote
1answer
36 views

For non-parametric regression which one has better interpretation and properties, GAM or quantile regression?

As in the topic. I want to interpret data for which I have no clues about the distribution. It's neither count, percentage, continuous. I don't want any transformations. Instead I would like to ...
1
vote
0answers
10 views

Rate of convergence of variance of kernel averages

I'm reading Hansen's (2008, p. 729) Theorem 1 where he bounds the variance of averages of the form $$\hat\Psi(x)=\frac{1}{Th}\sum_{t=1}^T Y_t K\bigg(\frac{x-X_t}{h}\bigg)$$ given that $\{(Y_t,X_t)\}_{...
1
vote
0answers
16 views

Is there a nonparametric regression to analyze repeated measures

I get involved in a study with only 40 subjects, and each subject has repeated measures at 6 months, 12 months, and 24 months. Originally I supposed that the total sample size 40x3 = 120 should be ...
1
vote
0answers
11 views

In Gaussian Process regression is there a way to force the prior slope to be positive when using a linear kernel?

I'm doing Gaussian process regression on some data $X$ with low sample size, using a squared exponential kernel. From domain knowledge I know that outside the range of my data the regressed function ...
0
votes
0answers
21 views

How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?

I have 2 input variables, $X_1$ and $X_2$ that affect output variable $Y$. I can run experiments where I modify the inputs and measure what happens to the output. Now, if $X_1$ and $X_2$ were binary, ...
2
votes
0answers
20 views

A comparison of the global optimal binwidth and local optimal binwidth of the histogram estimator

Suppose we have $X_1, \dots, X_n$ to be an i.i.d sample with unknown pdf $f(x)$ and cdf $F(x)$, and define $\hat{f} (x)$ to be the histogram estimator. We also define its Mean Integrated Square Error ...
1
vote
1answer
33 views

Nonparametric approach for regression with a quadratic fit

I'm trying to figure out which nonparametric test I should run on my data. My data has residuals that are not normal, so I cannot run a linear regression unless I log transform it. However, log ...
0
votes
0answers
63 views

Data Normalization for Time Series Forecasting using Nonparametric Model

What are the specialities of applying locally weighted learning for time series forecasting? I am trying to apply a nonparametric model ($K$-NN Regression) to forecast daily load curve (entire time ...
0
votes
0answers
104 views

Using Mean Cumulative Function (Nelson-Aalen) to assess drug efficacy

I have a large medical retrospective longitudinal dataset of electronic health records. An individual is identified by an ID. A medical event or drug prescription event is identified using a code and ...
1
vote
2answers
3k views

What is the cost/loss function of K nearest neighbors?

I am able to visualize how KNN works. Essentially take avg of the k nearest train neighbors for regression problem. However every ML algorithm optimizes a cost/loss function for example: Linear ...
1
vote
0answers
34 views

Nonparametric regression on data with known noise parameterization

What's the best way to regress on data for which we don't have a parameterised generative model (e.g. an arbitrary non-smooth continuous signal, that can be regressed in model-free ways with splines, ...
2
votes
0answers
35 views

Nonparametric regression with missing data

Let $(y_i,x_i,b_i)$ be data at hand, where $y_i$ is a response variable, $x_i$ is covariates, and $b_i$ is an indicator for missing: if 1, then $y_i$ is observable, 0 otherwise. Then, under missing at ...
1
vote
0answers
49 views

Finding the overlap of two Bivariate Non Parametric models using R

I need a few suggestions on some methods to try or to be pointed in the right direction. I will start off describing the big picture. I have little stats background. I come from physics and astronomy. ...
2
votes
1answer
2k views

Time complexity of leave-one-out cross validation for nonparametric regression?

From Artificial Intelligence: A modern approach: Most nonparametric models have the advantage that it is easy to do leave-one-out crossvalidation without having to recompute everything. With a k-...
2
votes
1answer
57 views

What is the density of $X$ under fixed design?

We observe an i.i.d. sample $(X_1, Y_1), \ldots (X_n, Y_n).$ Let $m(x) = E(Y|X=x),$ $\sigma^2(x) = \operatorname{Var}(Y|X=x)$ and let $f(\cdot)$ be the density of $X.$ Under some regularity ...
1
vote
0answers
211 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
5
votes
0answers
298 views

Is calculating a moving average a good way to approximate k-nearest neighbor regression?

Given i.i.d samples (x1,y1), ... (xn,yn) such that yi = f0(xi) + $\epsilon$i, i = 1,... n for some f0 Suppose I want an estimate $\hat{f}$ of f0 using k-nearest-neighbors regression in the ...
0
votes
0answers
97 views

Examples of using MCMC for GP regression

This is a reference request. I am in a position of needing to use MCMC to do Gaussian process regression for a project. I have used MCMC before, and I have used GPs before, but never together. It ...
1
vote
0answers
93 views

What is the basic difference between Sieves, Series and Splines estimators?

As far as I know, Sieve Estimators consists in a broader class of estimators for a function g(x) lying in a space of functions G. The estimation basically consists in choosing the function that best ...
1
vote
0answers
160 views

What is the nonparametric equivalent of the Weighted-Least-Squares-Regression?

I am struggling to find the nonparametric equivalent of the WLSR. In brief, I need a nonparametric regression method which allows to assign different weights to data according to the uncertainty. I ...
0
votes
0answers
115 views

approximate a nonparametric CDF in R

I have two vectors of same length. The first vector is a collection of realization from an unknown random variable. The second vector is the distribution computed at each particular realization. A ...
0
votes
1answer
177 views

Skewed Response Variable a lot zeroes

I am trying to build a non-parametric model to predict pure premium of an insurance policy. Pure premium is simply the expected claim amount of an insurance policy.The problem is, most insurance ...
2
votes
0answers
136 views

Is it appropriate to apply ordinal regression to a continuous response “as is” (with no ties)?

The goal is to run non-parametric general linear model with no restrictions on the design (e.g. not limited to one-way ANOVA as in Kruskal-Wallis test; quantitative factors should be admissible, etc) ...
2
votes
1answer
535 views

How to choose between parametric and non-parametric regressions? [closed]

I am new to nonparametric regressions. What tests should one perform to choose a non-parametric regression model over a parametric one(Or vice versa)? Let's assume in our analysis we have a continuous ...
3
votes
0answers
314 views

Options for non-linear or non-parametric count data regression in R?

I am trying to move from a current parametric Poisson regression to a non-parametric count regression and would appreciate views on the best way to do this. Current state of analysis I have a time ...
0
votes
1answer
389 views

How to fit nonparametric mixtures of regression models in a statistical program? [closed]

I am interested in estimating a nonparametric finite mixture regression model, explained for example here: https://methodology.psu.edu/media/techreports/09-93.pdf But I don't know with which program ...
3
votes
1answer
95 views

Why instrumental variables? Or: why not nonparametric regression?

Usually instrumental variables are introduced as a means to solve the problem $E(u|X)\neq 0$ in the model $Y = X'\beta + u$. This may happen if we omit important variables from the covariate vector $X$...
2
votes
0answers
177 views

Which non-parametric multiple-regression methods are computationally efficient with respect to the number of regressors?

I did some regression in R with random forests and got some decent results, $1-\sum{|e_i|}/\sum{|y_i-\bar{y}|}=0.692$, but I want to do better than this. Through my research, I have concluded that the ...
1
vote
1answer
336 views

Reporting the results of Nonparametric regression using kernel weights

I am wondering how I can present the results of nonparametric regression. I performed the nonparametric tests using R, and R package 'np'. The commands used for this are freq <- npreg(Respno ~ ...
1
vote
0answers
111 views

Linear Smoothers and cross validation. That is easy, but is it?

Consider the nonparametric regression problem $Y_i=m(x_i)+\epsilon_i$. Let $m_h$ be the estimator based on smoothing parameter $h$ $$\widehat{\mbox{MSE}}(h) = n^{-1} \sum_{i=1}^n \Big( \frac{Y_i-\...
1
vote
0answers
75 views

MSE of Non-parametric Regression (Theoretically)

I'm working through Computational Statistics, Second Edition by Givens and Hoeting and I cannot reproduce a result stated in the book. We're considering a nonparametric regression problem where the ...
2
votes
0answers
209 views

Question about the answer to “Local polynomial regression: Why does the variance increase monotonically in the degree?”

I appreciated Marco's elegant answer explaining why the variance of a local polynomial regression increases monotonically in the degree. However, in the end of the proof, I find difficult to calculate ...
8
votes
1answer
696 views

Rank and z-transform instead of Wilcoxon?

Andrew Gelman in a recent post in his blog suggests using a rank, transforming the rank to a z-score, and then using parametric tests and tools instead of performing non-parametric tests. I never ...
1
vote
1answer
1k views

R programming language: meaning of 'weights' parameter in library method 'loess'

I use the library method loess of the R programming language for non parametric data fitting. The dataset is two-dimensional. I have not found any proper documentation of the method parameter ...
3
votes
0answers
44 views

upper bound for expected maximum of difference of two kernel-Estimations

I'm searching for an upper bound for a function like $$ E\left[ \max_{x \in R} \left( \frac{ \sum_{i=1}^n K(\frac{x-X_i}{f(x,X_1, \dots X_n)}) \cdot Y_i } { \sum_{i=1}^n K(\frac{x-X_i}{f(x,...
4
votes
1answer
941 views

Local polynomial regression: Why does the variance increase monotonically in the degree?

How can I show that the variance of local polynomial regression is increasing with the degree of the polynomial (Exercise 6.3 in Elements of Statistical Learning, second edition)? This question has ...
1
vote
0answers
552 views

Why is locally weighted regression is a nonparametric method? How is it implemented in R?

I'm wondering where does the "nonparametric" label of locally weighted regression like LOESS or LOWESS comes from, i.e. why they are nonparametric methods? Also, I would like to know in general how ...
4
votes
1answer
62 views

Non parametric estimators for noisy functions

Suppose there is a function $f(a,b,c,\ldots)$ of $M$ variables (fixed numbers, not random variables). Add some Gaussian noise to this function: $$ g(a,b,c,\ldots) = f(a,b,c,\ldots) + \varepsilon(a,b,...
8
votes
1answer
491 views

Cross validation with nonparametric smoothing regressions

When I use regression models I feel leery of defaulting to an assumptions of linear association; instead I like to explore the functional form of relationships between dependent and explanatory ...