# How to estimate an unobserved variable?

Suppose I have a time series $$\{X_t\}_{t=1}^N$$ that is corrupted by Gaussian noise: $$Y_t = X_t + \epsilon_t,$$ where $$\epsilon_t \sim N(0, \sigma^2)$$, and we actually observe $$\{Y_t\}$$.

We have a deterministic model for $$\{X_t\}$$, such as $$X_t = f(t, \theta)$$, and we completely know the form of the function $$f$$ and even $$\theta$$. Up to the error introduced by the noise, we have a deterministic and causal model for how the process $$\{Y_t\}$$ is generated. E.g., we have a piece of machinery which generates $$\{X_t\}$$ but since we actually built the machine, we know the form of $$f$$. The Gaussian noise $$\epsilon$$ is simply noise introduced by our sensors and is easily estimated. (We can't actually write down $$f$$ in parametric form, but we can show you a directed acyclic graph representing $$f$$.)

Now, suppose something goes wrong with the machine, but we don't know exactly what. All we know is that when the machine is working, $$\{Y_t\}$$ is a stationary stochastic process, but now it is non-stationary and appears to be increasing. (E.g., temperature of the machine should be stable, but now its suddenly getting much hotter over time). We know that there is degradation in the system that produces $$\{Y_t\}$$, so we postulate that the signal is now being generated by the following process:

$$Y_t = X_t + \delta_t + \epsilon_t$$ where $$X_t$$ and $$\epsilon_t$$ are the same as before, but $$\delta_t$$ is an unobserved (latent?) variable for the corruption. We cannot observe $$\delta_t$$ except for its effects on $$Y_t$$. If $$\delta_t = g(t)$$ for some function $$g$$, we have no idea what $$g$$ would look like other than its an increasing function.

Question: Can we model $$\delta_t$$? How? What is this type of modeling called? Can we do it non-parametrically? When effects are observed "second hand" but not directly? I think this is related to factor analysis somehow, but that is not my area.

From what you are saying, in

$$Y_t = X_t + \delta_t + \epsilon_t$$

$$Y_t$$ and $$X_t = f(t, \theta)$$ are known, with $$\epsilon_t$$ being noise term and $$\delta_t = g(t)$$ is an unknown, latent variable. Then to find $$\delta_t$$, you need to model

$$Y_t - X_t = \delta_t + \epsilon_t$$

So basically, you need a time-series model for $$Y_t - X_t$$. The choice of time-series model in here is the same as choice of time-series model for any other time-series data.

• Thank you. I started to realize this was a solution shortly after I made my post. What about a more general case where $f$ is potentially unknown? Supppose you had a good model for $f$ before the introduction of $\delta$, would the procedure change after the fact? Aug 21 '19 at 6:25
• @Marcel if you have two unknown functions of time Y = f(t) + g(t), then they are not separable, you'll only be able to estimate Y = h(t) = f(t) + g(t), unless you can restrict them somehow (e.g. tend and seasonality, of different functional form).
– Tim
Aug 21 '19 at 10:12