# T-consistency vs. P-consistency

Francis Diebold has a blog post "Causality and T-Consistency vs. Correlation and P-Consistency" where he presents the notion of P-consistency, or presistency:

Consider a standard linear regression setting with $K$ regressors and sample size $N$. We will say that an estimator $\hat\beta$ is consistent for a treatment effect ("T-consistent") if $$\text{plim} \ \hat\beta^k = \frac{\partial E(y|x)}{\partial x_k},$$ $\forall k=1,\dots,K$; that is, if $$\left( \hat\beta_k−\frac{\partial E(y|x)}{\partial x_k} \right) \xrightarrow{p} 0,$$ $\forall k=1,\dots,K$. Hence in large samples $\hat\beta_k$ provides a good estimate of the effect on $y$ of a one-unit "treatment" performed on $x_k$. T-consistency is the standard econometric notion of consistency. Unfortunately, however, OLS is of course T-consistent only under highly-stringent assumptions. Assessing and establishing credibility of those assumptions in any given application is what makes significant parts of econometrics so tricky.

Now consider a different notion of consistency. Assuming quadratic loss, the predictive risk of a parameter configuration $\beta$ is $$R(\beta)=E(y−x′\beta)^2.$$ Let $B$ be a set of $\beta$'s and let $\beta^∗\in B$ minimize $R(\beta)$. We will say that $\hat\beta$ is consistent for a predictive effect ("P-consistent") if $$\text{plim} \ R(\hat\beta)=R(\beta^∗);$$ that is, if $$(R(\hat\beta)−R(\beta^∗)) \xrightarrow{p} 0.$$ Hence in large samples $\hat\beta$ provides a good way to predict $y$ for any hypothetical $x$: simply use $x′\hat\beta$. Crucially, OLS is essentially always P-consistent; we require almost no assumptions.

<...>

The bottom line: In sharp contrast to T-consistency, P-consistency comes almost for free, yet it's the invaluable foundation on which all of (non-causal) predictive modeling builds. Would that such wonderful low-hanging fruit were more widely available!

Questions:

1. What are the conditions under which P-consistency holds?
2. Simple counterexample(s) where P-consistency does not hold
3. Does presence of T-consistency imply presence of P-consistency?
• Note to self: Hayashi "Econometrics" section 2.9 "Least squares projection" could be relevant. Specifically: That is, under Assumption 2.2 (ergodic stationarity) and Assumption 2.4 guaranteeing the nonsingularity of $\mathbb{E}(XX^\top)$, the OLS estimator is always consistent for the projection coefficient vector, the $\beta^*$ that satisfies the orthogonality condition (2.9.5). – Richard Hardy Feb 9 '19 at 17:57
• P(redictive?)-consistency seems to be close to calibration, see calibration. – kjetil b halvorsen Sep 12 at 1:19

The way these terms are defined suggests that, for "T-consistency", one cares about whether $$\hat{\beta}$$ is close to the true $$\beta$$, whereas "P-consistency" is concerned with whether $$\hat{y}$$ will be close to $$y$$.

What are the conditions under which P-consistency holds?

What is defined as "predictive risk" is just the mean square error of a linear prediction. "P-consistency" just means consistent estimation of the best linear predictor $$x' \beta^*$$, in time series language.

The OLS estimate $$\hat{\beta}$$ consistently estimates $$\beta^*$$, under very general assumptions. This is because $$\hat{\beta}$$ is just a sample version of $$\beta^*$$, and you just need the sample moments that enter into $$\hat{\beta}$$ to converge to population moments entering $$\beta^*$$. In other words, one needs LLN to hold (same for the consistency of any method of moments estimator).

The conditions needed are just weak stationarity (so that $$\beta^* = \frac{Cov(x,y)}{Var(x)}$$ is defined) and, e.g. strong-mixing type of conditions like $$\alpha$$-mixing with no restriction on the mixing rate and existence of enough moments (usually 4 would do it).

Therefore, "OLS always identifies the best linear prediction", in more econometric vernacular.

Simple counterexample(s) where P-consistency does not hold

There may be examples of weakly stationary processes for which strong-mixing conditions do not hold and LLN does not hold. In such cases, the probability limit of OLS $$\hat{\beta}$$ would not exist and "P-consistency" does not hold.

For your spurious regression example, $$\beta^*$$ is not defined, as the processes are not stationary. In talking about "P-consistency", one already implicitly assumes stationarity so $$\beta^*$$ is defined.

Does presence of T-consistency imply presence of P-consistency?

In the context of linear models, "T-consistency" means $$\hat{\beta}$$ estimates the "true" $$\beta$$ where regressors are exogenous $$E[\epsilon x] = 0$$. But exogeneity just means that true $$\beta$$ is equal to $$\beta^*$$.

So, since "T-consistency" and exogeneity are empirically the same (the latter is a sufficient condition but this conflation is standard), yes would be a fair answer.

Estimating the conditional mean (T-consistency) is a stronger requirement than estimating the linear projection (P-consistency).

Addendum---Examples where P-consistency does not hold

Consider the case of the trivial regression on an intercept only (where $$y = \beta$$). In this case, P-consistency is equivalent to LLN. If we can find a (strictly stationary, say) time series $$x_t$$ for which LLN does not hold, then P-consistency does not hold for the regression $$x_t = 1 + u_t.$$

Here is one such series. Take two i.i.d. series $$x_{1,t}$$ and $$x_{2,t}$$ such that $$E[x_{1,t}] = 0$$ and $$E[x_{2,t}] = 1$$. Define $$x_t = \begin{cases} x_{1,t}, & \text{for all t, with probability \frac12} \\ x_{2,t}, & \text{for all t, with probability \frac12} \\ \end{cases}.$$ Then $$E[x_{t}] = \frac{1}{2}$$ but $$\frac{1}{n}\sum_{t=1}^n x_t \rightarrow \begin{cases} 0 & \text{with probability \frac12} \\ 1 & \text{with probability \frac12} \\ \end{cases}.$$ Therefore P-consistency does not hold. This is the simplest example of a strictly stationary non-ergodic series. (Under ergodicity, one has the ergodic LLN.)

Next we introduce an error term to get a linear regression model. Let $$\epsilon_t \stackrel{i.i.d.}{\sim} (0, \sigma^2)$$, $$(\epsilon_t)$$ and $$(x_t)$$ be independent, and $$y_t = \beta x_t + \epsilon_t.$$ Let $$\|\cdot\|$$ denote Euclidean norm on $$\mathbb{R}^n$$. Then $$\| \frac{1}{n} (\hat{y}_n - y )^2 \|^2 = (\frac{1}{n} \sum_{t=1}^n x_t \epsilon_t)^2$$ which does not have almost sure or probability limit, for similar reasons: $$\frac{1}{n} \sum_{t=1}^n x_t \epsilon_t \rightarrow \begin{cases} 0 & \text{on a set A with P(A) = \frac12} \\ 1 & \text{on a set A^c with P(A^c) = \frac12} \\ \end{cases}.$$ Therefore P-consistency does not hold.

Empirical Comment

Any strictly stationary non-ergodic time series takes on similar form as $$(x_t)$$ above, after relaxing the i.i.d. assumption on $$x_{1,t}$$ and $$x_{2,t}$$ to just strict stationarity. Empirically one might say that such processes have "very long memory". This is in contrast with a mere long memory series, which can be ergodic. For example, the fractional Gaussian noise (FGN) is ergodic and has long memory (what makes it long memory is that the variance of its partial sums grows like $$n^{\alpha}$$, for $$\alpha > 1$$). In particular, ergodic LLN holds for the FGN.

To the extent that one believes the long-memory property defines the upper boundary of dependence-over-time observed in data series, perhaps one empirical take-away from the above example is that P-consistency can always be assumed to hold.

(Long memory property was first observed in Nile river data by Hurst. It has also been suggested that stock returns could have long memory---see, e.g. here. I don't know of any empirical example where a stationary non-ergodic model has been entertained---inference seems impossible when LLN does not hold.)

• No disagreement with previous answer by @CarlosCinelli. – Michael Jun 13 at 8:57
• Thank you and +1. Could you spell out any concrete counterexample? I am not comfortable with mixing, unfortunately. – Richard Hardy Jun 13 at 9:17
• @RichardHardy Will try to see if it would be possible to add one later. (Such an example does not change the empiricist's take away that OLS is "always" consistent, under stationary.) – Michael Jun 13 at 9:20
• @RichardHardy Since you quoted an ergodicity condition in the comment, for now let me also say that mixing is something like ergodicity in that it describes a kind of "asymptotic independence", but ergodicity is a notion restricted to strictly stationary series. So you could replace mixing with the ergodicity condition you quote, which gives the ergodic LLN. – Michael Jun 13 at 16:56
• This makes me think a bit of collinearity where the estimates of $\beta$ may have high variance but the estimate of $y$ is still good (low variance). Although in that case it is still both t-consistent and p-consistent. I find it difficult to imagine a situation that is p-consistent but not t-consistent. – Sextus Empiricus Sep 10 at 6:00

This post by Francis is conflating causal concepts with statistical (regression) concepts.

First, the definition of a treatment effect is incorrect: a causal effect is defined in terms of a causal model, not in terms of the observed joint probability distribution.

The quantity $$\frac{\partial E[y|x]}{\partial x_k}$$ is not a treatment effect, it is the marginal change in the expected value of $$y$$ when we observe a change in $$x_{k}$$. For any linear conditional expectation function, this quantity is always the population regression coefficient of $$x_k$$.

A treatment effect estimand would make reference to interventions or counterfactuals,such as $$\frac{\partial E[y_{x}]}{\partial x_k}$$ in counterfactual notation, or $$\frac{\partial E[y|do(x)]}{\partial x_k}$$ in $$do()$$ notation.

Thus, his statement, “the distinction between P-consistency and T-consistency is clearly linked to the distinction between correlation and causality” is total nonsense.

1) By definition, OLS is the solution to minimizing $$R(\beta)$$, so it always holds under the usual standard assumptions that you have i.i.d samples from a stationary (well behaved) process.

2) Your RW example is not really valid, because a random walk is not stationary. I’m saying this because otherwise a counterexample could be just: “Imagine you have n observations, each coming from different arbitrary distributions.” In the RW case, you could estimate a specific time point regression coefficient consistently with OLS if you had infinite samples from that time point. Thus, to make this question meaningful, you would need to properly define the constraints, such as in Wasserman’s post, where he confined the coefficients to a l1 ball, and let the dimension of the ball grow with the sample size. And note this has nothing to do with causality.

3) As explained, in the usual set-up P-consistency always holds regardless of whether $$E[y|x]$$ is linear, and thus whether the OLS coefficients corresponds to $$\frac{\partial E[y|x]}{\partial x_k}$$ (which is not the definition a "treatment effect", as Francis incorrectly states).

• Thank you for the very relevant an interesting points! These are very helpful for my understanding. At the same time, your answers to the specific questions are not quite sufficient / not concrete enough. In (1), is i.i.d samples from a stationary process the condition? Could you be more specific? Usually, I encounter either i.i.d. or stationary, not both (though i.i.d. implies stationarity). In (2), the example seems to be valid precisely because it is a counterexample which I am asking for. It specifically violates the stationarity assumption. Do you think this does not make sense? – Richard Hardy Nov 6 '18 at 6:11
• In (3), could you elaborate a bit more? Could you specify the conditions under which T-consistency implies P-consistency? Thanks again! – Richard Hardy Nov 6 '18 at 6:12
• @RichardHardy thanks for pinging, I had forgotten about it! I have a deadline coming up, but after will come back to complement the answer. – Carlos Cinelli Jan 16 '19 at 7:31
• @RichardHardy back here! So, regarding (i) my main point is that by definition the population regression coefficients are always identifiable. You do not need i.i.d necessarily, the most well known case is when you have heteroskedastic data, or some forms of autocorrelation, in which the data is not iid, yet consistency still holds. I believe there is a paper spelling out necessary conditions for consistency under a general set-up, I will try to find it. But my main point is that since OLS is the minimizer of $R(\beta)$ in the population, consistency generally holds barring pathological cases. – Carlos Cinelli Feb 4 '19 at 23:40
• @RichardHardy Regarding (iii) if the conditional expectation is linear, the population regression coefficients are also the derivatives of the conditional expectation. Thus, in this case consistency of $\beta$ implies consistency of the derivatives of the conditional expectation, the only extra assumption is linearity of $E[Y|X]$. In the non-linear case, Francis question is underspecified, since to start we must define better what the target is (the derivative at one point X = x^*, the derivative averaged over the other X?). – Carlos Cinelli Feb 4 '19 at 23:55
1. --- ? ---
2. P-consistency will not hold when regressing two independent random walks on each other. In this setup, the OLS estimator converges to a random variable rather than the true parameter value (which is zero).
(Then w.r.t. 1., we need some sort of stationarity or moment conditions for P-consistency.)
3. --- ? ---
• Besides you I don't think many people on this site have the background or knowledge required to answer such an advanced question ! Is there some form of "canonical" or base question regarding risk consistency ? Here are a set of notes which are very good from Tibshirani, in particular sparcity and linear regression stat.cmu.edu/~ryantibs/statml/lectures – Xavier Bourret Sicotte Sep 13 '18 at 6:12
• @XavierBourretSicotte, I will take this as a compliment, thank you. However, I know quite a few people on this site who are clearly more knowledgeable and/or experienced than I am. I am sure more than one of them could answer the question if they tried hard enough. The problem might be that either they are not interested in the question or they do not have the time to try hard enough (or simply they have not seen the question – the view count is quite low). Regarding a canonical question regarding risk consistency, I do not know the answer, sorry. – Richard Hardy Sep 13 '18 at 7:22