2
$\begingroup$

I've been going through time series material of late, trying to re-invent myself as a practitioner in the field. Until I got to the point of actually trying to Matlab some models, I had never run across the characterization of models as "conditional mean". This term is pervasive throughout the entire documentation set; from what I can tell, it carries a particular significance in terms of modelling.

I haven't been able to wrap my head around what this significance is. It is explained as the expression of the expectation of a variable of interest, conditional upon the values of other variables. All the time series models that I've seen (indeed, all regression models) fall under this definition. So I'm not sure how the term classifies models of interest, or identifies a subset, or identifies features that have modelling implications. Nothing I've found on line makes it any clearer.

Thanks if anyone can provide a clarification, elaboration, or explanation of this. My question arose in the context of Matlab (it just happens to be what I am using), but it is a conceptual question that isn't necessarily tied to Matlab. It has been posted to: What is the meaning characterizing a model as "conditional mean", https://stackoverflow.com/questions/30406645/what-is-the-meaning-characterizing-a-model-as-conditional-mean, and http://groups.google.com/forum/#!topic/comp.soft-sys.matlab/zQK8yScPFkk

$\endgroup$
  • $\begingroup$ Examples of models for conditional variance (rather than conditional mean): ARCH and GARCH models (and their numerous offsprings), stochastic volatility models. $\endgroup$ – Richard Hardy May 27 '15 at 19:26
4
$\begingroup$

The first time you usually see this concept is in linear regression.

Let's suppose that we have a hypothetical relationship between two random variables that looks like

$$ Y = aX + b + \epsilon $$

where $\epsilon$ is some unobserved variable that is symmetric about it's mean of zero. Now suppose we take some observations of $X$ and $Y$ and want to use them to figure out a good guess at the parameters $a$ and $b$ in the relationship. Well, the value of $X$ clearly influences the value of $Y$, so we may want to proceed in two stages:

  • Given that I know a value of $X$ already, what can I deduce about the distribution of $Y$?
  • What is the distribution of $X$?

A first cut at the first question is asking:

If I know a value of $X$ already, what is the mean of $Y$?

And this is the conditional expectation of $Y$ given $X$, usually notated $E(Y \mid X)$ (and referred to as some as the conditional mean). This is the quantity that linear regression aims to estimate, because:

$$ \begin{align} E(Y \mid X) &= a*E(X \mid X) + bE(1 \mid X) + E(\epsilon \mid X) \\ &= aX + b + 0 \\ &= aX + b \end{align} $$

Figuring out what's going on in each of these steps, and possibly what I left unsaid that is needed to make the calculation go through, is a nice way to test your understanding of the concept.

In time series, you often build autoregressive models, which assume a relationship like:

$$ Y_k = a_0 + a_1 Y_{k-1} + a_2 Y_{k-2} + \epsilon_k $$

for example. Here the same kind of reasoning tells you that you would really like to know the mean of $Y_k$, given that you already know the values of $Y_{k-1}$ and $Y_{k-2}$. This is called $E(Y_k \mid Y_{k-1}, Y_{k-2})$. The comma on the right of the condition symbol means and in this context.

I get what you explained, and I guess my confusion is...what regression or projection model of interest isn't conditional mean? In other words, how is that characterization adding any information about the model that one is interested in? If there were models of interest that don't fall under that characterization, I can understand how that characterization conveys information. But the only characterization I can think of that isn't conditional mean is a pure noise source. But as I said, I'm new in the field, so I think I must be missing an entire body of models (of interest).

Notice that I got to the conditional mean by asking myself:

What's something simple I would want to know about the distribution of $Y$, given that I already know $X$?

Well, there are plenty of things I could ask:

  • Whats the mean of $Y$ given $X$?
  • What's the median of $Y$ given $X$?
  • What's the $90$'th percentile of $Y$ given $X$?

All these are valid questions, and there is a model for each of them. To get at the conditional mean of $Y$ given X, you minimize the expected squared error:

$$ E(Y \mid X) = arg\,min_f E\left( (Y - f(X))^2 \right)$$

To get at the conditional median you minimize the expected absolute error:

$$ Median(Y \mid X) = arg\,min_f E\left( \left| Y - f(X) \right| \right)$$

To get at the 90'th percentile you do something else clever. The last two cases are covered by a theory called quantile regression. It has nice properties like resilience to outliers and asymmetry in the same way that the median does.

$\endgroup$
  • $\begingroup$ I get what you explained, and I guess my confusion is...what regression or projection model of interest isn't conditional mean? In other words, how is that characterization adding any information about the model that one is interested in? If there were models of interest that don't fall under that characterization, I can understand how that characterization conveys information. But the only characterization I can think of that isn't conditional mean is a pure noise source. But as I said, I'm new in the field, so I think I must be missing an entire body of models (of interest). $\endgroup$ – StatSmartWannaB May 22 '15 at 23:31
  • $\begingroup$ @StatSmartWannaB I added an example of a non-conditional mean model. $\endgroup$ – Matthew Drury May 22 '15 at 23:54
  • $\begingroup$ Huh. OK. I get that one can characterize the heck out of the uncertainty due to the nuisance parameter. I've just never seen such a preoccupation with that particular detail in the context of regression or time-series. I thought I was missing something big whenever I saw mention of conditional mean models, but if it's just a reminder that in other contexts, you could characterize processes by other metrics, I guess I can translate "conditional mean model" to "regression or time series". In other words, I won't worry too much about it. Thanks for clearing that up, Matthew. $\endgroup$ – StatSmartWannaB May 23 '15 at 1:20
  • $\begingroup$ P.S. I realize I abused the term "regression", since it's way more than ordinary least squares. I looked up the quantile regression. Interesting. For some reason, it smells like rank order statistics, though if I think more than a minute about it, the connection is quite tenuous. $\endgroup$ – StatSmartWannaB May 23 '15 at 1:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.