Even though I've studied Probability and Statistics, my knowledge of Bayes stops at the famous formula. So, when it comes to modern Bayesian computational methods, I'm just able to understand answers to questions like this one, but I'm still missing the most important point: playing around with some Bayesian tools, I've noticed that models parameters are gradually updated as new information is disclosed.
Let me explain better what I would like to understand: when we're dealing with time series analysis and prediction, we often have a huge sample but we also have the task to select a proper window width to estimate our model parameters; this is because we are burdened by the usual trade-off between long and short term memory, for which:
- using a large sample would probably mean a very stable estimate but unable to adapt quickly to structural changes in the time series;
- using a small sample would probably mean an unstable estimate but able to adapt to structural changes.
The simplest example of this trade-off is a moving average of the time series, but consider more convoluted examples like a function which estimates the parameters of a simple $AR(1)$ model and the application of such function through a rolling window: the autocorrelation parameter's time series, $\rho$, is going to have lower variance in the larger window width case than in the smaller one.
(In the figure below I've produced an example by estimating $\rho$ from a rolling window $AR(1)$ model on a time series to show that the values obtained can be very different according to the window width).
After some trials with MCMC tools and Bayesian statistic applied to models parameters, I've seen things like this:
If the sample is a time series, my intuition suggests me that the density of
beta in the figure above comes from a parameter that varies over time. This "variation over time" should simply be a time series of
beta that is gradually updated while the prior distribution is plugged into the Bayesian tool. Then parameters distribution is sampled from MCMC instead of being represented by a single value coming out from the solution to some minimization problem (I'm thinking about OLS). From this point of view, Kalman filter is a Bayesian tool when used to solve dynamic linear models, because of the mechanism that updates the values of the parameters as soon as new information is disclosed.
- do Bayesian statistics tools behave like a rolling/expanding window parameter estimation when applied to a time series?
- Do Bayesian statistics behave like the Kalman filter when applied to a time series?
- If 1 and/or 2 are at least partially true, can Bayesian statistics and MCMC be considered tools that solve the aforementioned trade-off of time series analysis window width?
- Could you please provide a simple example in
Rfor this problem: show the time-varying parameters that describe the mean and the standard deviation of your favourite time series as Bayesian statistics would yield them by using MCMC. You can use any package and any time series you want.