Skip to main content
33 votes

Trend in irregular time series data

Rather than try to decompose the time series explicitly, I would instead suggest that you model the data spatio-temporally because, as you'll see below, the long-term trend likely varies spatially, ...
Gavin Simpson's user avatar
19 votes

Explain what is meant by a deterministic and stochastic trend in relation to the following time series process?

The deterministic trend is one that you can determine from the equation directly, for example for the time series process $y_t = ct + \varepsilon$ has a deterministic trend with an expected value of $...
Rob's user avatar
  • 532
19 votes
Accepted

Why Time series decomposition is performed

Time series decomposition helps us disentangle the time series into components that may be easier to understand and, yes, to forecast. In principle, yes, you can see pretty much everything in the ...
Stephan Kolassa's user avatar
15 votes
Accepted

Why is it valid to detrend time series with regression?

You're astute in sensing that there may be conflict between classical assumptions of ordinary least squares linear regression and the serial dependence commonly found in the time series setting. ...
Matthew Gunn's user avatar
  • 22.5k
15 votes

Is Naive Bayes becoming more popular? Why?

I'd be cautious about over interpreting Google trends. Here's naive bayes (blue) vs. k-means (red). What does it mean? I can make up a story that common variation is due to machine learning classes ...
Matthew Gunn's user avatar
  • 22.5k
13 votes
Accepted

Shall we use log(diff(x)) or diff(log(x))?

Lets's have at look at both options. diff(log(x)) diff(log(x)) calculates relative changes. This also takes care of ...
ndevln's user avatar
  • 353
11 votes
Accepted

Why this formula for 5 year trend?

The formula is the linear regression of the series (whatever the variable is), which I will call $y$, on time ($x$), giving units-of-$y$ per year. This is then divided by average $y$ and the result ...
Glen_b's user avatar
  • 284k
11 votes

Can a timeseries with a clear trend be considered stationary?

From the help page: The general regression equation which incorporates a constant and a linear trend is used and the t-statistic for a first order autoregressive coefficient equals one is computed. ...
Stephan Kolassa's user avatar
10 votes
Accepted

stochastic vs. deterministic trend in time series

Deterministic Trend $$ y_t = \beta_0 + \beta_1 t + \epsilon_t $$ where $\{\epsilon_t\}$ is white noise, for simplicity. Same discussion applies to the case where $\{\epsilon_t\}$ is a covariance-...
Michael's user avatar
  • 3,328
10 votes
Accepted

How do I present averages from different sample sizes across years?

The usual thing to do here would be to include "error bars" around your sample average giving a confidence interval for the true average from the sampled data each year. For binary data you ...
Ben's user avatar
  • 127k
9 votes
Accepted

Polynomial Regression - why do Excel coefficients differ from R's?

Try poly.model <- lm(y ~ poly(x, 5 , raw = TRUE)) ...
Antoni Parellada's user avatar
9 votes

Do I have to add the seasonal effect and trend back to ARIMA forecast?

No, you do not need to remove trend and/or seasonality before fitting an ARIMA model. These models can handle certain types of trends and certain types of seasonality by themselves, or by including ...
Chris Haug's user avatar
  • 5,850
8 votes
Accepted

Is the sum of trends of two time series the trend of the sum of the time series?

Is the sum of trends of two time series the trend of the sum of the time series? - as a general question, it depends: if the estimator is linear in the data then yes, but in general, no. On the ...
Glen_b's user avatar
  • 284k
8 votes
Accepted

Can stats regarding the Coronavirus outbreak tell us anything at all?

Don't let the perfect be the enemy of the good Many of the issues you have raised are perfectly reasonable concerns. Having said that, it is important to distinguish between cases where reported data ...
Ben's user avatar
  • 127k
7 votes

What does "linear-by-linear association" in SPSS mean?

As a previous reply mentioned, yes it is and the technical description is at SPSS's support page: https://www-304.ibm.com/support/docview.wss?uid=swg21477269 This is a useful statistic for those who ...
MikeG's user avatar
  • 116
7 votes

Trend and Breakout detection in time series

There are several solutions to your problem. There are two forms of outliers: Additive outlier (also called as pulses) Level shifts (also called as break in trend). I'm assuming you would need step ...
forecaster's user avatar
  • 8,335
7 votes
Accepted

Deseasonalizing data with fourier analysis

The term you're looking for is "trend and seasonality decomposition of time series". Google this. There are many approaches. If you really have only 100 points then Fourier will not work very well. ...
Aksakal's user avatar
  • 61.6k
7 votes
Accepted

When is first differences for time series trend removal appropriate to use?

When is first differences for time series trend removal appropriate to use? If you are dealing with cumulative sums of stationary series, differencing is a natural transformation to perform. ...
Richard Hardy's user avatar
7 votes
Accepted

how can i simulate with arima.sim drift, intercept and trend

Deterministic Trend If your drift intercept is $c$, you can just add the function $c t$ to the zero mean process. Code: ...
Taylor's user avatar
  • 20.8k
7 votes

Can you suggest a novice method for detrending and deseasoning time series to find relationship between two variables?

I wanted to expand on my comment: So let's simulate a series which exhibits a basic trend, seasonality, and reacts to an exogenous variable. (I am working in python) ...
Tylerr's user avatar
  • 1,562
6 votes

Random Forest Regression and trended time-series

Just change the variable you are trying to predict to the difference in the dependent variable. As the other posts point out, the random forest will not know how to treat time variables that occur ...
Anders Christiansen's user avatar
6 votes
Accepted

predict seasonality and trend combined, better approach?

There are several methods and models for this kind of analysis, for example: exponential smoothing, ARIMA time series models or structural time series models. The topic is too broad to be covered here....
javlacalle's user avatar
  • 11.7k
6 votes

Deseasonalizing data with fourier analysis

Classic auto-regressive models can handle cycles! Going way back, Yule (1927) and Walker (1931) modeled the periodicity of sunspots using an equation of the form: $$y_{t+1} = a + b_1 y_t + b_2 y_{t-1}...
Matthew Gunn's user avatar
  • 22.5k
6 votes

Detecting trend reversal in a time series

With realistically noisy signals, this is not a simple problem. For example, you write a heuristic rule would say that a trend reversal has occurred if the value has been trending in a direction ...
Ami Tavory's user avatar
  • 4,590
6 votes

Why is it valid to detrend time series with regression?

Basic least-squares type regression methods don't assume that the y-values are i.i.d. They assume that the residuals (i.e. y-value minus true trend) are i.i.d. Other methods of regression exist ...
G_B's user avatar
  • 600
6 votes
Accepted

Exponential regression trendline does not match data

A function of the form $m_Y = \exp(a+bx)$ (where $m$ represents some conditional population coefficient of interest, like a mean, geometric mean, or a median, perhaps, depending on how your error term ...
Glen_b's user avatar
  • 284k
6 votes
Accepted

Finding p for trend in R - testing the trend across an ordinal categorical variable

You have to transform your factor predictor to an ordered factor by, e.g. mydata$ord_predictor <- ordered(mydata$predictor) Then in the output of your model, ...
utobi's user avatar
  • 11.8k
6 votes

ETS (error, trend, seasonal) formulation

This is explained in my 2008 Springer book (robjhyndman.com/expsmooth), Section 2.5.2. Here is an excerpt. The general model involves a state vector $\textbf{x}_t = (\ell_t, b_t, s_t, s_{t−1}, \dots, ...
Rob Hyndman's user avatar
  • 57.1k
5 votes

predict seasonality and trend combined, better approach?

If your data are simple, and you are just toying around, the decomposition into seasonal and overall trend should already be pretty good. But if you want to dig deeper, there is a more formal approach:...
Winks's user avatar
  • 3,711
5 votes

How to characterize abrupt change?

This inference problem has many names, including change points, switch points, break points, broken line regression, broken stick regression, bilinear regression, piecewise linear regression, local ...
Jonas Lindeløv's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible