# Estimating variance of an unobserved component

Consider $N$ time-series $x_{i,t}$ coming from the following data-generating process:

$$x_{i,t}=\beta_i y_t + \xi_{i,t} + \varepsilon_{i,t},$$

where $y_t$ is a common component among all the series, it is i.i.d. in time with mean 0 and variance $\sigma^2$; $\varepsilon_{i,t}$ is i.i.d. in time and across series with mean 0 and variance $\sigma_i^2$. For every $t$, $\xi_{i,t}$ satisfies the following property

$$\sum_{i=1}^N \xi_{i,t} = 0$$

At the same time $\xi_{i,t}$ is identically distributed in time and across series with mean 0 and variance $\sigma_{\xi}$.

None of the parameters ($\beta_i, \sigma, \sigma_i, \sigma_{\xi}$) are known; $y_t$, $\xi_{i,t}$ and $\varepsilon_{i,t}$ are unobservable. One observes a sample of length $T$ of $x_{i,t}$, $\{x_{i,t}\}_{i=1,t=1}^{i=N,t=T}$, only.

Questions:

1. What are possible ways to estimate $\sigma_{\xi}$ and $\sigma_i$? What are more robust/statistically efficient ways to estimate them?

2. Is there any way to infer the value of $\sum_{i=1}^N |\xi_{i,t}|$ for every $t$?

Any thoughts are welcome. For practical considerations assume $\sigma_i$, $\sigma$, $\sigma_{\xi}$ are of similar order of magnitude and $\beta_i \in (0,2)$.

• I suspect the model is not identified, i.e. there exist different combinations of parameters who all yield the same likelihood. You would need to fix some of the parameters (i.e. normalize) to be able to estimate the remaining ones uniquely. Oct 22, 2017 at 10:07

If you rewrite your model in state space form: $$\mathbf{x}_{t}= \mathbf{B} y_t + \left[\begin{array}{c} \xi_{1,t} \\ \xi_{2,t} \\ \xi_{3,t} \\ -\sum_{i=1}^{n-1} \xi_{i,t} \end{array}\right] + \varepsilon_{t}.$$ $$\left[\begin{array}{c} \xi_{1,t} \\ \xi_{2,t}\\ \xi_{3,t}\\ \end{array}\right] = \left[\begin{array}{ccc} .9 & 0 & 0 \\ 0 & .9 & 0\\ 0 & 0 & .9\\ \end{array}\right] \left[\begin{array}{c} \xi_{1,t-1} \\ \xi_{2,t-1}\\ \xi_{3,t-1}\\ \end{array}\right] + \left[\begin{array}{c} \epsilon_{1,t-1} \\ \epsilon_{2,t-1}\\ \epsilon_{3,t-1}\\ \end{array}\right]$$ then

1. you can estimate all unknown parameters with the EM algorithm or ML techniques with general purpose optimizers, and
2. you can do (2) with a Kalman Smoother, using the parameters you estimated from (1).

NB: This is one particular model that satisfies your requirements. There are other options available,and in particular, there are other dynamics you can choose for the states that satisfy your requirements. However, if you choose a state transition kernel that is nonlinear, then (1) and (2) will not be true, and you will require techniques that are more complicated.

For starters, and this would still allow you to use (1) and (2), it might be worthwhile to replace the $.9$s with different parameters, or assume they are unknown as well.

Also I am assuming you have a typo and mean to say that you observe $\{x_{i,t}\}$ and $\{y_{i,t}\}$.

Here is a wrong solution strategy.

1. Construct series $\hat{y_t} = \sum_{i=1}^N x_{i,t}$. For large $N$ this is a good proxy for $y$ up to a proportionality coefficient.

2. Run OLS regressions: $x_{i,t} = \hat{\alpha_i} +\hat{\beta_i} \hat{y_t}+ \hat{\zeta}_{i,t}$

3. Sample variance $V[\hat{\zeta}_{i,t}]$ will be a consistent estimate for $\sigma_i^2+\sigma_{\xi}^2$

4. Notice that $$\sum_i V[x_{i,t}]-V\left[\sum_i x_{i,t}\right]= V[y_t]\left(\sum_i \beta_i^2-\left(\sum_i \beta_i\right)^2\right)+\sum_i V[\xi_{i,t}]$$

5. Take the sample counterpart of the above expression by substituting sample variances for $V[x_{i,t}]$, $V\left[\sum_i x_{i,t}\right]$, $V[\hat{y_t}]$, and estimates of $\beta_i$ obtained at step 2 to calculate an estimate for $\sum_i V[\xi_{i,t}]$. Divide it by $N$ to get an estimate $\hat{\sigma}^2_{\xi}$.

6. Subtract $\hat{\sigma}_{\xi}^2$ from the sample variance $V[\hat{\zeta}_{i,t}]$ to obtain $\hat{\sigma}^2_i$.

• This answer is actually wrong. Estimated $\beta_i$ contain noise. So the sample counterpart of $\sum V[\xi_{i,t}]$ obtained at step 5 actually equals sum of sample variances of the residuals $\hat{\zeta}_{i,t}$. Oct 23, 2017 at 21:28