# Questions tagged [smoothing]

Smoothing methods in data analysis, like splines or kernel smoothers, also regression smoothers like lowess.

399 questions
Filter by
Sorted by
Tagged with
18 views

### Adjust winrates based on number of games played using Bayesian estimator

I am sorting players of an online video games based on their winrate, but I'm trying to use the number of games they scored that winrate on. The idea behind it is that even though 9 wins for 1 losse ...
31 views

### Meaning/Definition of Smoothing

I am self-studying Vector Generalized Linear/Additive Models (mostly) via Yee, T. W. (2015) "Vector Generalized Linear and Additive Models: With an Implementation in R." This book has many ...
• 153
12 views

### Forward-Backward Algorithm for Autoregressive HMMs

I am currently studying HMMs, and covered the Forward-Backward Algorithms as well as the smoothing and filtering process. Recently, we were posed a question on Autoregressive HMMs which I've been ...
• 83
83 views

### How to detect a plateau at the end of a short time series?

In exercise testing (spiroergometry), participants' oxygen uptake is measured continuously while they engange in physical exercise that progressively intensifies. The measurements are taken at regular ...
• 31k
21 views

### Smoothing spatial data for privacy issues

I have a dataset with about 6000 observations related to municipalities. The structure of the dataset includes four columns: the municipality code, the number of businesses in the municipality, the ...
• 327
46 views

### How to create a GAM with interation term for a soap film smoother

I am interested in creating a GAM that contains a soap film smoother that can also vary in time. Below is an example using synthetic data. I start by creating data with two spatial signals, which are ...
• 3,712
119 views

### GAM(M): "Shape" vs "wiggliness"?

I'm currently going over how to interpret certain terms of a generalized additive mixed model (GAMM). I'm having a hard time differentiating between "shape" and "curvature". ...
• 1,641
48 views

### Aggregation of smoothing terms in GAMs

I have fitted a GAM model by using the R package mgcv via the expression model <- gam(formula, data, family = binomial()) ...
• 331
1 vote
62 views

### Smoothing out target variable for spiky demand forecasting

I am trying to predict ambulance demand for the next hour, for a city area in the USA, based on previous demand, weather, large people gatherings, and similar spatio-temporal factors - using Machine ...
257 views

### Significance testing on Generalized Additive Mixed Models (GAMMs) - mgcv::gam

I am wondering whether anyone has any insight on comparing the performance of two GAMMs? Specifically, I want to compare two models: A "nested model", one with a smoothed age term ...
47 views

### When smoothing using a cubic regression factor smooth in R using mgcv’s gam model, how do I set the knots and k for each level of the factor smooth?

I’ve fit a Poisson generalized additive model (GAM) to predict the number of deaths given exposure using the R library mgcv and its ...
• 41
81 views

### Smoothing a transition matrix

I have a dataset containing observations at time t and t+1 of ratings A (best) to E (worst) and Default. I want to use transition matrices to predict future ratings t+2, t+3, etc The transition matrix ...
172 views

### mgcv gam P-value vs plotted confidence intervals

I am trying to figure out how to explain the apparent discrepancy between P-values and plotted confidence intervals in mgcv. See for example the plot below that comes from a model that considers the ...
• 455
25 views

### How to detect moments of state change in boolean time series?

Let $A = (X_1, ... X_n)$ be a series consisting of long periods of ones interrupted with shorter periods of zeros, like $$A = 111111111 00000001111111111100000011111111111111.$$ The plot below (left) ...
• 291
10 views

### How does Kneser Ney estimate ngrams with BOS without dividing by zero?

The recursive formula for (unmodified) Kneser Ney smoothing is (per Jurafsky08 3.40) $P_{KN}(w_i | h) = \frac{\text{max}(c_{KN}(h\ w_i) - d, 0)}{\sum_v c_{KN}(h \ v )} + \lambda(h) P_{KN}(w_i | h)$ ...
• 1,386
16 views

### Assessing probability that one set of measurements extends the other with Kalman smoother

I have two sets of N-dimensional measurements following each other with a certain time gap in between. Let's name those sets $A$ and $B$, respectively. All observations have constant Gaussian white ...
23 views

### Why would you smooth a logical variable in a GAM?

I just used the gam.scope function in the R-package gam to create possible scopes of each explanatory variable of my model to ...
142 views

### Is it possible to use smooth functions as part of a nonlinear regression?

Background I am fitting nonlinear regressions with a single response and two predictors. I know the relationship between the response and each of the predictors, but I do not know how they interact. I ...
• 18.9k
64 views

### Inferences with Filtered and Smoothed state estimates from a tracking problem

I am working on a synthetic tracking problem in a two-dimensional space, where the start and end positions are known and noisy measurements of the state variables are given at discrete time steps. As ...
427 views

### How does lowess handle gaps in time series?

I have been using the lowess smoother to calculate trends for time series data for a while now but until now my data was always without gaps. I now have to work with data where there are quite large ...
• 177
1 vote
181 views

### Understanding Identifiability problem of multiple smooths in GAMs/Additive models

I am a beginner into additive models and GAMs, but have good enough knowledge of linear regression. I was going through this wikipedia article to understand more, but can't seem to understand the ...
1 vote
188 views

### Smoothing of GPS tracks - remove noise and stop-go clusters

I know there are several posts about this, but I could not find exactly what I need. I have GPS track data (from an underwater vehicle) for short intervals of 1 second (time-stamps on data). The data ...
• 11
1 vote
99 views

### Functional Data Analysis with Missing Data

I am trying to analyze a dataset containing several observations for n = 27. Unfortunately, not every participant answered the question every time, i.e., when plotting the graph it has holes in it. I ...
• 11
174 views

### GAM partial response plot interpretation

I've made the gam in the code below (in R), but I'm struggling to interpret the results. Specifically, the partial response plots for all but one of the variables is linear, and the CI lines cross in ...
63 views

• 101
1k views

### Smoothing spline seems to fit too precisely?

I am trying to fit some time series data to a smoothing spline in R. However, it seems like the spline is fitting the data too perfectly, meaning overfitting. I was trying to figure out what settings ...
• 1,522
2k views

### How to decide which variables to smooth in GAM

When specifying it's formula GAM has s function for smoothing. Let's say I want to fit mtcars. I can do the following: Approach 1: ...
• 141
193 views

### How to choose the smoothing method and factors that best capture the trend in data?

I have a time-series data that I will be using to forecast the future using machine learning algorithms such as LSTM. Prior to the forecast, I need to smooth the data to remove the noise and capture ...
114 views

### What reasons beyond interpretability are there to use additive models over a complex, multivariate smoother?

Let's adopt for the second the notation from the R package mgcv. ...
• 1,373
1 vote
71 views

### Can do smoothing after making time-series data stationary

Can do smoothing after making time-series data stationary? And is it useful to do smoothing before making data stationary?
119 views

### Determining spline basis dimension using Wood's statistical test

In Simon Wood's book Generalized Additive Models (2nd ed.) on page 243, he describes the following procedure for checking that the basis dimension is too small: Fortunately informal checking that the ...
• 1,373
1 vote
955 views

### Change y-axis value in R GAM plot from smoothed to actual response variable

I know similar questions have been asked before, but I've looked through them all and I'm still confused, in part because the other examples seem to be more complicated, so thought I'd throw mine up. ...
• 25
1 vote