Forecasting average values with varying number of observations Every day $t$ we observe several (independent) realizations of a variable $X_t$. This variable is the sum of a time dependent mean value and white noise:
$$X_t=\mu_t+\epsilon_t$$
You can assume $\epsilon_t$ has a fixed variance over time, is Gaussian and independent of everything. The underlying time series $\mu_t$ could be forecasted with usual time-series methods. 
But $\mu_t$ is not directly observable. Instead we observe:


*

*$a_t$ the average observed value for $X_t$ over day $t$

*$n_t$ the number of observations for this day


The number $n_t$ can vary very much. The goal is not to forecast it but to take it into account to forecast $\mu_t$. Typically, when $n_t$ is small, we observe a noisy average. When $n_t$ is big, $a_t$ is very close to $\mu_t$ (the standard deviation of $a_t-\mu_t$ is $\frac{\sigma}{\sqrt n_t}$)
Do you know/think of a way to take $n_t$ into account when using forecasting methods? (Possibly very simple ones: assuming $\mu_t$ is just a random walk)
 A: The idea is to use the Kalman filter. An easy to read (but quite complete) introduction is available here: How a Kalman filter works, in pictures. It is essentially Gaussian Bayesian inference.
Assume $\mu_t$ is just a random walk with Gaussian step of known variance $1$. I explain how to implement the Kalman filter in this simple case.
At each time $t$ the filter saves its knowledge about $\mu_t$ as a Gaussian distribution for  $\mu_t$ with mean $m$ and variance $s^2$. When you move forward into the future, you loose information because $\mu_t$ varies randomly with variance $1$. The filter is updated as such:
$$m\leftarrow m
\\
s^2\leftarrow s^2+1\\$$
When you observe $x$ a realization of $X_t$, the filter does a Bayesian inference resulting in:
$$m\leftarrow \frac{\frac{1}{\sigma^2}x+\frac{1}{s^2}m}{\frac{1}{\sigma^2}+\frac{1}{s^2}}
\\
s^2\leftarrow \frac{1}{\frac{1}{\sigma^2}+\frac{1}{s^2}}\\$$
You can manage $n$ observations as a single update replacing $x$ by the average value $a_t$ and $\sigma^2$ by $\sigma^2/n_t$.
In other words, when you move forward into the future, the uncertainty $s^2$ increases. After each observation the uncertainty $s^2$ decreases and the forecast $m$ is updated with weights depending on the present uncertainty and the number of observations.
Red: observations. Blue: $m$ after observations

A: Interesting question ! If you model the average as a function of N and find that the variance of the conditional average is related to N then this might suggest a Box-Cox value of 0 . Why don't you post data for 100 consecutive days and I will try to investigate your assertion.
