Questions tagged [loess]
LOESS (or LOWESS) stands for locally weighted scatterplot smoothing. It is a form of local (k-nearest neighbor) kernel regression
95
questions
1
vote
1answer
37 views
R: “family” and “degree” specification in loess fitting
I can't understand the difference between the possible specifications of the family option in the loess command in R. This is ...
0
votes
0answers
7 views
Why is loess decomposition not a good way to find trend?
I have been working with time series where i decompose time series to trend and residuals, I have seen kaggle etc, but no one uses loess to decompose time series and get trend separately, although ...
0
votes
0answers
12 views
How should smoothing be applied to train and test set?
I want to smooth my data through lowess regression, how should lowess be applied, should it be applied separately on train and test set? I applied lowess on lagged sales value and then divided data in ...
0
votes
0answers
23 views
Trend estimation: strange behaviour of loess
I am trying to estimate the trend of a time series. I have used a non-robust loess smoothing with a window size of 30 points. Looking at the raw data, it is clear that there is no increase at the end ...
0
votes
0answers
7 views
Mitigating edge effects in Loess
I have some time series data which are expected to show an effect (of a national Covid-19 lockdown.) Recent figures confirm the suspicion both qualitatively and when plotting loess lines, of varying ...
0
votes
0answers
21 views
Automatic selection of smoothing parameter with time series
Cross validation allows to automatically select a reasonable smoothing parameter for a given data set, avoiding underfitting and overfitting. Cross validation requires independent data, an assumption ...
10
votes
2answers
124 views
Statistics on LOESS smoothing
I'm trying to make sense of a data set which contains thousands of independent measurements of intensity in the form of a scatterplot. These measurements are dependent on two major variables: ...
1
vote
0answers
20 views
Bootstrapping GAM LOESS models with Multiple Predictors [closed]
I have a data set with multiple predictors and am using a GAM model in conjunction with LOESS. I am trying to replicate this but include the bootstrap process as well. The following code is the ...
1
vote
0answers
61 views
STL decomposition of a daily time series (only business days)
I'm currently working with a daily (business days) time series which has a monthly seasonality and an overall positive trend over the last two years. I want to estimate the error component of the time ...
0
votes
1answer
42 views
Taking the mean of several data entries on same date to make one
I am analyzing the number of likes of several influencers over time and I want to plot it to see if there is a trend.
Some influencers post several times on the same date, and then overall several ...
0
votes
0answers
22 views
What is the seasonal trend lowess model in time series?
I am trying to understand how the trend line is produced automatically from software I am using for time series (iNZight). A textbook I have says "a local weighted regression line is fitted to a ...
1
vote
0answers
53 views
What smoothing parameter makes sense for a LOESS calibration curve?
I am creating a calibration curve to asses the fit of a logistic regression.
Does it make more sense to use the local or global optimum smoothing parameter for the LOESS line?
The orange line uses ...
0
votes
0answers
160 views
How to Assess Linearity assumption of logit in logistic regression
In Applied Lineare Regression, (Hosmer, Lemeshow, Sturdivant 3rd ed.) Ch. 4, they present "Purposeful Selection." Part of step 5 is to assess the validity of the linearity assumption of the logit vs ...
2
votes
1answer
1k views
How is slope calculated in a calibration plot?
I am using logistic regression with white cell count and temperature as predictors and hospital admission>3 days as the outcome of interest. I'm using the rms package in R to assess calibration (curve ...
9
votes
2answers
2k views
Why so much difference in SE areas in these graphs
I am using ggplot with Python for showing regression/correlation. With method='lm' (means "linear model"), I get following graph:
And with method='loess', I get following:
The width of SE area is ...
3
votes
1answer
275 views
Should I use a confidence interval or a prediction interval around the LOESS fitted curve?
A Freakonometrics blog post shows how to use a LOESS regression of the residuals of a logistic model on the predicted values of the logistic model to assess the linearity of the predictors used in the ...
3
votes
3answers
1k views
What is the difference between Local Linear Regression (LLR) and Locally Estimated Scatterplot Smoothing (LOESS)?
I've looked into nonparametric regression packages in R and Python and came across two estimation methods that are relevant for my problem (i.e. replicating the semiparametric estimation in Carneiro, ...
2
votes
0answers
55 views
Loess preprocessing for robust quadratic regression?
It seems to me that we could fit a quadratic curve to the predictions of a LOESS curve in order to obtain a parametric model approximating the non-parametric LOESS model. Is this something that is ...
0
votes
2answers
232 views
What is represented by the y-axis in a loess smoothing curve?
I'm working on the Titanic data and I plotted local regression curves for a couple predictors.
...
2
votes
2answers
3k views
Rules of thumb for partial residual (component + residual) plots as diagnostics for linearity?
Here are the standard R diagnostic plots of a multiple linear regression model that includes an autoregressive term at lag-1 (i.e. AR(1)). I have logged & z-scored my input data.
Ben Bolker says ...
1
vote
0answers
59 views
How to detect an increase in a loess model fitted value at the end
Sorry if the question is trivial, but I'm not finding a proper idea for this issue.
I'd like to find if a series of fitted value of a loess is increasing in the end.
I'm working with some data like ...
4
votes
1answer
2k views
Decomposing a time series with some zero values
There are many techniques to decompose a time series into trend, seasonal, and remainder components. I was wondering if these techniques can be applied without worry to time series which have some ...
0
votes
1answer
59 views
Machine Learning to filter similar loess curves from a large dataset
I think I have a good dataset to apply machine learning.
I have a bunch of scatter plots and I generated a loess for each. Each loess is stored as a series of Y coordinates that correspond to X ...
2
votes
1answer
3k views
Difficulty plotting a loess smooth: what causes these problems?
I'm very new at this and I don't actually understand the differences between the plotting methods, but loess seems to be giving me the most informative graphs, ...
0
votes
1answer
116 views
STL function in R: is it possible to make it backward looking?
I would like to smooth modeling input time series data using the stl function in R.
Based on my understanding, the function performs a local regression using data ...
2
votes
0answers
206 views
Why does stl() decomposition require integer frequency?
I need to decompose and forecast weekly series with around 10 years of data. In this data leap years play an important role so I need the have non-integer frequency, frequency = (365.25/7)
By reading ...
3
votes
2answers
439 views
Expectation of Median of Absolute Random Variables
Let $X_1, X_2,..., X_n$ from $N(0,\sigma^2)$.
What I want to get is not $E(median(|X|))$ , but $E(median(|X_1|,|X_2|,...,|X_n|))$
Reason why I need it is because I'm studying LOWESS, and in using ...
2
votes
0answers
271 views
Seasonal weights of a time series: stl function versus subtracting average values
Say that we are dealing with a time series of 25 years of daily measurements of a variable with seasonal cycles, e.g. it reaches higher values in winter than in summer.
The objective is to compute ...
2
votes
1answer
604 views
How are the local polynomials in LOESS regression joined to make a smooth curve?
I am trying to understand LOESS regression. I've read the Wikipedia and Wolfram articles on it, and the R help for the loess() function.
I think I understand now how the local polynomials are derived,...
2
votes
0answers
363 views
goodness of fit / likelihood of data given model for loess
Question
How can I compute the likelihood of a set of data points given a LOESS fit (of the same and/or other data points)?
Alternative
Are there other (better?) ways to assess how 'good' a LOESS ...
4
votes
2answers
3k views
Could the equation of the curve provided by LOESS be obtained?
Considering the LOESS fitting done with the following R code:
...
6
votes
1answer
1k views
Use loess regression with many zero values
I have measuments of vegetation coverage on Y plotted against surface height (and hence flooding frequency) on X. The vegetation often has two herb layers, which are estimated seperately. If only one ...
3
votes
1answer
337 views
Should I capitalize GAM, MARS and LOESS?
I have to mention GAM (generalized additive models), MARS (multivariate adaptive regression splines), and LOESS in an academic paper.
I think GAM and MARS are capitalized in most cases, but not sure ...
23
votes
1answer
2k views
Explanation of what Nate Silver said about loess
In a question I asked recently, I was told that it was a big "no-no" to extrapolate with loess. But, in Nate Silver's most recent article on FiveThirtyEight.com he discussed using loess for making ...
1
vote
0answers
455 views
Prediction Interval for Loess Forecast
I have consulted this question on the basis of prediction intervals for loess (How to calculate prediction intervals for LOESS?). However, I am unaware of whether it is proper or possible to use ...
2
votes
1answer
1k views
Confidence interval of first derivative of a loess smooth
I have plotted the evolution of parameter vs time. I would however prefer to plot the change in these parameter vs time. Here I have seen how to predict a confidence interval for a loess smooth and ...
1
vote
0answers
523 views
Weights in Loess
I have a good working knowledge of how the loess model works but am curious as to how weights work in conjunction with the model. Obviously, this method weights locally, but many statistical packages (...
2
votes
1answer
3k views
How does a LOESS model do its prediction?
I understand the theory behind LOESS, but how does it do prediction without coefficients?
I'd like to use LOESS prediction, but need to be able to explain it.
4
votes
2answers
5k views
Fitting a smoothed curve to a noisy data
I have a variable with sales data over time. It is very noisy at a disaggregate level but if you look at it as a whole, you can see a smoothing curve that follows a polynomial pattern. Is there a way ...
0
votes
0answers
227 views
Unexpected low values in LOESS regression
The following plot is generated using the scatterplot function from R's car package.
The ...
0
votes
4answers
565 views
Non-linear terms in logistic regression?
I have a data set with binary response and several numerical and categorical predictors. I'm looking for ways to test if some of the numerical predictors are non-linear. I've read about lowess and ...
3
votes
1answer
604 views
Behaviour of neighbouring points in LOESS smoothing when data is not uniformly distributed
The purpose of loess is to create an 'average' value of the response for any $x$, by using points in the region of $x$ to create a local regression line.
From what I understand, picking the ...
2
votes
0answers
996 views
Locally weighted regression vs. splines
What's the pros/cons of splines approaches compared to locally weighted regression approaches for the purposes of (a) scatter plot smoothing and (b) prediction?
Obviously, in the case of prediction I ...
3
votes
1answer
762 views
Calculating bias and variance in a LOESS fit
I have datasets that can form several different curvy patterns between the dependent and independent variables. The 'true' relationship likely depends on a large number of factors that aren't easily ...
15
votes
1answer
6k views
GAM vs LOESS vs splines
Context: I want to draw a line in a scatterplot that doesn't appear parametric, therefore I am using geom_smooth() in ggplot in <...
6
votes
0answers
255 views
When to choose GAM over LOESS? [duplicate]
I am smoothing the relationship between a binary variable y and a continuous variable x. For this I've looked into both GAM and ...
1
vote
1answer
166 views
Time series LOESS prediction performs better on aggregate data?
I'm using the stl function in R (https://stat.ethz.ch/R-manual/R-devel/library/stats/html/stl.html) that uses loess to decompose time series and make predictions.
When the time unit is one month, the ...
4
votes
0answers
1k views
Robustness Weights in LOESS behaving strangely
I've been playing around with writing my own LOESS module in Python (2 reasons: first, I wanted the practice, and second, the implementation in statsmodels doesn't ...
2
votes
1answer
1k views
LOESS for subset of data
I want to analyze the performance of two classifiers. For that I have a dataset with 30000 observations that each have an independent variable interactions and a ...
2
votes
1answer
161 views
Loess in gene enrichment analysis using DESeq
Often MA plots (Bland-Altman style representations of gene enrichments) are corrected using the LOESS method. Will a LOESS correction significantly affect the results of gene enrichment analysis? More ...
