On estimating ARIMA models on artificially made time series data

Question

For each day, I observe my variable, y(t), for a period of 12 hours. In order to understand the data and make predictions, I want to put together these data and make a long timeseries data. Now, if I fit an AR(1) model to the data, or even do kalman filtering on data as explained later,it would mean that my first reading on a day depends on the last reading of the previous day, which is meaningless in my context.

Question how can I stop the last value of previous day to influence the first observation of new day?

What I've done:

So I have put NA s of the same length of each observation data (12) in between any two days to preserve the seasonality. But I have trouble understanding the consequences of this:

1. upon seeing these NA's, will arima() ignore the NA's, practically doing the same thing I'm trying to avoid?
2. will it try to interpolate the values, which again means constructing the same series?

Now if it is represented in state-space format, the system would just update the state equation (interpolating 'NA's?). But after updating it for 12 times before reaching the next non-missing value, would the state vector be a reasonable value?

All the computations are done in R.

EDIT, clarification: Assume for each day, I'm collecting data from 1 pm to 12 am. Ideally, for each day, what I want to get at is a model of type $y(t) = \phi y(t-1) + \beta y(t-12) + v(t)$. now If I have a continuous timeseries, then $y(13) = \phi y(12) + ...$ where $y(12)$ is the last value from previous day. Now, in my context, it is meaningless for data on 1pm to depend on 12 am value of last day, but it's reasonable for 3pm data to be dependent on 2 or 1pm, or on 3pm data from previous day(s).

Also assume that for the first 2 hours of each day, I don't need to forecast them.

Answer 1

I understand what your after. In the original version you said you desired an AR(1) and/or a Kalman filter type model. Lets begin with an AR(1).

AR(1)

Let $y_{t,i}$ denote the observed value at time $t$ on day $i$ and say you have $n$ days total. Suppose that for all $i$, the series $\{y_{t,i}\}^{t=12}_{t=1}$ are stationary and follow the same AR(1) process, so $$ y_{t,i}=\phi_0 + \phi_1y_{t-1,i}+u_{t,i},\;\; u_{t,i} \stackrel{iid}{\sim}N(0,\sigma^2) $$ You could alternatively allow for change in the constant or volatility for each day,...but that is besides the point.

For all $t>2$ we know the conditional distribution of $y_{t,i}$, that is we know $$ y_{t,i}|y_{t-1,i},..y_{1,i} \sim f(y_{t,i}|y_{t-1,i};\phi_0,\phi_1,\sigma^2) \equiv N(\phi_0 +\phi_1y_{t-1,i}\,,\,\sigma^2) $$ But we do not know this conditional distribution at $t=1$ because we do not have a $y_{0,i}$ and cannot use $y_{12,i-1}$.

However, we can still use the unconditional distribution of $y_{t,i}$, i.e. we know $E[y_{t,i}]=\frac{\phi_0}{1-\phi_1}$, and $VAR[y_{t,i}]=\frac{\sigma^2}{1-\phi^2_1}$. So for all $i$ we know $$ y_{1,i} \sim f(y_{t,i};\phi_0,\phi_1,\sigma^2)\equiv N\bigg(\frac{\phi_0}{1-\phi_1}, \frac{\sigma^2}{1-\phi^2_1}\bigg) $$

As such we can define the exact likelihood as $$ f(y_{1,1},...,y_{12,1},y_{1,2}...,y_{12,n}|\phi_0,\phi_1,\sigma^2) =\prod_{i=1}^{n}\bigg[f(y_{1,i};\phi_0,\phi_1,\sigma^2)\prod_{t=2}^{12}f(y_{t,i}|y_{t-1,i};\phi_0,\phi_1,\sigma^2)\bigg] \quad(1) $$

You can estimate the parameters vector $(\phi_0,\phi_1,\sigma^2)$ by maximizing the log of $(1)$ or using it as a likelihood in MCMC.

Alternatively, you could find the parameters with least squared regression but then you would have to through out every $y_{1,i}$. Usually this would not be a big deal because you would only need to through out one observation, but here it would result in getting rid of 1/12$^{th}$ of the data which is probably a big deal.

Your edit makes things a little more complicated, because you introduce the idea of more than one lag and seasonality. It is the same sort of idea as above, however, with more complicated models defining the exact likelihood is more challenging. http://econ.nsysu.edu.tw/ezfiles/124/1124/img/Chapter17_MaximumLikelihoodEstimation.pdf is a good start for getting a feel of the exact AR(p) likelihoods, you will still have to adjust it to what you want though.

Kalman Filter

It is the same idea with a Kalman filter. Without getting lost in notation, define the likelihood of the Kalman filter as

$$ \prod_{i=1}^n\bigg[\varphi \bigg(\frac{y_{1,i}-\hat y_{1,i}}{\omega_{1,i}} \bigg) \prod_{t=2}^{12} \varphi \bigg(\frac{y_{t,i}-\hat y_{t,i}}{\omega_{t,i}} \bigg)\bigg] $$

where $\varphi$ denotes the standard normal pdf, and $\hat y_{t,i}$ and $\omega^2_{t,i}$ are the Kalman filter estimates of mean and variance for time $t$ of day $i$.

If you have used a Kalman filter before, you are aware that it requires an estimate of an initial state so that the kalman filter estimates for the first time period consist of an update of that initial state. Assuming the states are stationary, choosing the initial state is relatively simple as you can make it a function of your transition matrix and other parameters.

In this case, I believe you would have $n$ initial states so that for each $i$, $\hat y_{t,i}$ and $\omega^2_{t,i}$ are updates of the $i^{th}$ initial state. Assuming your data followed the same process for each day, I see no reason for all $n$ initial states to not be identical.