Questions tagged [smoothing]
Smoothing methods in data analysis, like splines or kernel smoothers, also regression smoothers like lowess.
319
questions
1
vote
0answers
27 views
STL decomposition of a daily time series (only business days)
I'm currently working with a daily (business days) time series which has a monthly seasonality and an overall positive trend over the last two years. I want to estimate the error component of the time ...
4
votes
2answers
40 views
Is smoothing an appropriate solution to deal with model diagnostics in a GAMLSS?
I have just recently started using GAMLSS models (after being pointed in that direction in this question), and I'm wondering wether it's 'legit' to use smoothing (i.e. cubic splines in my case) to ...
2
votes
1answer
36 views
Smoothing technique that takes into account subtotals
I have a simple task of fitting a smooth line with monthly points across six years. The line doesn't need to go through any specific point, but the yearly totals need to be specific values.
Is there a ...
0
votes
0answers
19 views
Biase of ASE estimation Kernel Regression
I'm trying to calculate the bias of the estimator $p(h)=n^{-1}\displaystyle\sum_{i=1}^{n}(Y_{j}-\hat{m}_{h}(X_{j})^{2}w(X_{j})$ of the averaged squared error. The result I find in the literature is ...
0
votes
0answers
9 views
What's the good way to find outliers in the continuous physical process measurements time-series? (picture attached) [duplicate]
What's the best way to find outliers in the time-series like the one attached?
Just using the knowledge that it is a measurement of a continuous physical process. The data is quite short (200-500 ...
2
votes
1answer
34 views
How to fit a smooth polynomial?
We have $N$ samples of the unknown function $f(x)$ on the finite interval $[a, b]$. The samples are subject to white noise of known variance. We want to approximate the function $f(x)$ by a polynomial ...
0
votes
1answer
18 views
Quantify smoothness of vector field
I have different vectors field and I want to characterize their "smoothness" or tendency to align, or like their heterogeneity. Here are two examples, a relatively smooth vector field on the ...
0
votes
0answers
14 views
How is the smoothing spline penalty computed in practice?
I'm digging into smoothing splines and finding good resources, but no one talks about how to actually calculate the penalty $\int \hat{f}^{"}(x)^2 dx$ in the standard smoothing spline loss:
Since <...
1
vote
1answer
64 views
What's the best way to find outliers in the time-series, encountering that it is a real-world mechanical process (process continuity)?
What's the best way to find outliers in the time-series, encountering continuity?
I attached two time-series that I'm interested to filter. One is less noisy, and one is a bit noisier. I'm mostly ...
2
votes
1answer
62 views
Smoothing algorithm for anomalies
To construct a plot, I'm looking for an algorithm which can handle inf and (very) negative values.
If I have infinity values everything is a line but not infinities.
Example
...
1
vote
0answers
28 views
Hyper-parameter optimization for regression to avoid overfiting/underfiting
I am using Penalized spline to smooth noisy data. Those splines are non parametric regression models which only rely on a smoothing parameter $\lambda \geq 0$ (which has to be chosen).
I would like to ...
0
votes
0answers
15 views
How choose smoothing function for Generative Additive Models? (GAMs)
I would like to know which strategies are used in practice to choose the correct smoothing function for each features in the GAMs, both for classification and regression tasks. I thought of plotting ...
1
vote
3answers
54 views
How to avoid the ending drop of the moving average curve?
I'm playing around with the built-in matlab movmean function which, using default options, creates the simple and centered moving average of given data (I'm pretty ...
1
vote
0answers
33 views
Smooth Curve Fit
I uses supsmu to fit a smooth curve to a scatter plot of data .
I am quite new to curve fitting, but my graph looks somewhat sinusoidal. What would you recommend in terms of next steps to determine a ...
0
votes
1answer
31 views
How to smooth an existing PDF?
I've generated a PDF of binned data using the python package binsmooth. The PDF is plotted in the following image:
I am trying to smooth the PDF so as to provide a more intuitive interpretation of ...
0
votes
0answers
15 views
Smoothing methods for unevenly sampled data
I have categorical, time-series data distributed in space. It is very noisy, but over the whole series there are big shifts in distribution - my goal is to see how these shifts progressed in space. So,...
1
vote
0answers
32 views
Multidimensional Smoothing spline in R
How create multidimensional smoothing spline for regression in R? Is there a function to implement thin-plate spline or tensor product?
I see the function ssanova() based on tensor product but I don't ...
2
votes
1answer
43 views
What does low rank smoother mean?
I read a paper on thin plate splines that stated several advantages. Thin plate splines:
Do not rely on ’knots’ as many other spline methods do
Are of isotropic character (i.e. unaffected by the ...
1
vote
0answers
15 views
Prediction with smooth.spline [closed]
I am a bit unsure how the newdata= argument is set in the prediction function. Here I have found the optimal degree of freedom to be 4 via 10-fold CV. I have fitted ...
0
votes
0answers
28 views
Smoothing spline with df chosen by 10-fold CV
So instead of using the (LOOCV) cv argument in smooth.spline(), I would like to use a block of 10-fold CV code to find the optimal degree of freedom.
I find that the smooth.spline model is somewhat ...
1
vote
0answers
28 views
What smoothing parameter makes sense for a LOESS calibration curve?
I am creating a calibration curve to asses the fit of a logistic regression.
Does it make more sense to use the local or global optimum smoothing parameter for the LOESS line?
The orange line uses ...
0
votes
0answers
35 views
Choosing between additive and multiplicative Holt-Winters Model
While using the Holt-Winters model for seasonality, I am unable to choose a better fit between additive and multiplicative models. I used to look at RMSE value and choose the one with the lower RMSE. ...
1
vote
1answer
39 views
Confused about the cubic regression basis in the R package mgcv
To understand the cubic regression basis constructed in the R package mgcv, I plotted out the 5 bases generated within [0,1]:
...
1
vote
0answers
24 views
Using prior information for a GAM smooth function to reduce standard errors
I have some data that I want to model in a GAM. However, there are few observations, generally leading to high standard errors.
...
0
votes
1answer
71 views
What is the correct mgcv syntax for interacting a smooth with an interaction of linear predictors?
I have a question about mgcv's formula syntax for interactions with smooths s() (I am actually using this syntax within ...
2
votes
3answers
122 views
How to avoid smoothing to 0 at edges of R density plot
When using the density function in R, it includes smooth transitions down to 0 at both ends of the data. Is there a way to prevent this? As a trivial example, ...
0
votes
0answers
24 views
Unexpected Y predicted value using Generalized additive model with smooth term on predictor in R
I have made GAM model about a relation between marine debris concentration (as Y variable) with beach feature and a distance from a point location to a river, port, tourism object and city (as X ...
0
votes
1answer
98 views
Smoothing a binned averages
I am trying to smooth some binned data. I have a discrete variable X which might best be thought of as time and a continuous variable Y. I want to know the average Y value for each value of X and this ...
1
vote
1answer
27 views
How to measure smoothness of inputs over outputs?
I know similar questions have been asked for time series data. But my question is a little bit different.
Consider that we have input dataset $X \in R^{N \times M}$, where $M$ is the dimension of ...
0
votes
0answers
20 views
is there any equation that can be inferred from the parameters estimated with the method smooth.spline function in R?
If the answer is yes, what is the equation?
of the smooth.spline function in R function where do I find the estimated coefficients for use in the equation?
Thanks for the help
1
vote
0answers
29 views
Smoothing 3D motion tracking data [closed]
I'm working on an application that tracks an object's position in 3D space through time. The hardware generates (x,y,z,t) data points, with a non-uniform sampling ...
2
votes
2answers
152 views
Smoothing histograms with kernel methods
I have a problem where I can receive as output, multidimensional counts in "histogram" form. I can also adjust the size of the bins I receive (i.e., many or few bins). I want to smooth the data and ...
0
votes
0answers
29 views
Smoothing methods for geographically aggregated data?
I have some Canadian census data with various statistics defined by dissemination block. These blocks are irregular in shape since their boundaries are based on the road network. I thought it would be ...
2
votes
0answers
24 views
Using Hierarchical Bayesian Network for Data Smoothing
I have a dataset that contains the mean speed at which 50 car models move, along with the standard deviation (based on an underlying database of sampled car speeds).
Using this mean and deviation, I ...
1
vote
1answer
85 views
How should I understand smoothing in functional data analysis from a modelling perspective (specifically for temperature data)?
The specifics of this question are that I am looking at daily maximum and minimum temperature from the GHCND data set obtained from NOAA's API and I view the temperatures observed at days in a year as ...
0
votes
0answers
93 views
Equation for smoothing spline from coefficients?
If I smooth a data vector with a smoothing cubic spline my understanding is that each ‘segment’ between knots should be representable as a cubic polynomial.
Is it possible to infer the equation of ...
5
votes
2answers
194 views
Forcing smoothness of regression coefficients
I'm building regression models on spectral datasets: the predictors are the intensites of signal at the different frequencies. In this case the intensities at close frequency values are highly ...
0
votes
1answer
55 views
Smoothing time series with non-constant variance
I have a discrete time series $x(t)$ $ t = \{0,\Delta t,2\Delta t\dots\}$ in which every point comes with a confidence value $c(t)$ arising from the measurements. You may think of is as the variance ...
1
vote
2answers
82 views
What are the exact factors used in smooth.spline by R?
What exactly is optimized mathematically when I use:
smooth.spline(x, y, lambda)
in terms of the integrated second derivative?
Is it
$$\min_{f\in C^2} \sum_{i=1}...
0
votes
0answers
154 views
Demmler-Reinsch basis for smoothing splines
I have seen some papers about using the so-called Demmler-Reinsch basis for smoothing spline because it is a basis for natural spline space and also Sobolev space. For example, these papers:
A ...
1
vote
0answers
284 views
Smoothing in a monotonically increasing manner
I have a number of curves that contain numbers from between 0 and 1. The curves should be monotonically increasing, but due to random noise, there may be some times where it is decreasing.
Is there ...
0
votes
0answers
63 views
predicting X values from smooth.spline
I have an existing smooth.spline object, and I wish to estimate X values for a set of new Y values. I see that ...
0
votes
1answer
56 views
Questions re. fitting a polynomial: Runge's phenomenon solutions
I have data on hospital treatment times. I would like to fit a polynomial to the data using least-squares.
In a previous question raised before I have already been advised against this, but ...
2
votes
2answers
99 views
Questions re. fitting a polynomial: smoothing, cross-validation, etc
I have data on hospital treatment times. I would like to fit a polynomial to the data. My data comes in 5 minute increments and it is very noisy. It looks like this:
I can aggregate to a higher ...
2
votes
1answer
39 views
How can I smooth data in 2D coordinates that has time-dependent error?
I have collected some GPS data from running over and around a hill many, many times.
The hill itself is about 9-10 meters high compared to the ground around it, although when I collected data, my ...
1
vote
2answers
478 views
What is the difference between smoothing and decomposition in time series?
i am bit new to time series modelling, currently i am trying to understand some basics. What is the difference between smoothing and decomposition in time series . I have gone through many materials , ...
0
votes
0answers
33 views
Matrix inversion in the smoothing splines [duplicate]
The question is about the matrix inversion in the smoothing splines.
Given observations (y1, x1), ..., (yn, xn) and a choice of $\lambda \ge 0$, the smoothing spline estimator, $\hat{f}_{\lambda}$ is ...
0
votes
1answer
158 views
Should I do detrending or smoothing first?
Does it matter which one I perform first? If yes, why?
Might be a simple question, yet I could not find an answer anywhere else.
3
votes
1answer
173 views
How does the penalized form of RSS (residual sum of squares) work?
In another word, how to reverse engineering the equation (5.9) by explain all the assumption and reasoning after the plus sign of (5.9) in Elements of Statistical Learning:
$$\text{RSS}(f,\lambda) = \...
1
vote
0answers
67 views
What does alpha in smoothing stand for
I want to apply Gaussian smoothing to a dataset and came across the smth.gaussian function in R. That besides the numerical input data requires two parameters:
...
