11
$\begingroup$

I'm looking for a paper which I hope exists, but don't know if it does. It could be a set of case studies, and / or an argument from probability theory, about why using cross-sectional data to infer / predict longitudinal changes may be a Bad Thing (i.e. is not necessarily so, but can be).

I've seen the mistake made in a couple of big ways: inferences were made that because richer people in Britain travel more, then as society gets richer, the population as a whole will travel more. That inference turned out to be untrue for an extended period - more than a decade. And a similar pattern with domestic electricity use: cross-sectional data implies big increases with income, that don't manifest over time.

There are several things going on, including cohort effects and supply-side constraints.

It would be very useful to have a single reference that compiled case studies like that; and / or used probability theory to illustrate why it is that using cross-sectional data to infer / predict longitudinal changes can be very very misleading.

Does such a paper exist, and if so, what is it?

$\endgroup$
  • 1
    $\begingroup$ I believe economists would think about these phenomena as a kind of general equilibrium effect. Stats people call this a violation of Stable Unit Treatment Value Assumption. I think the panel vs cross section issue is a bit of a red herring. $\endgroup$ – Dimitriy V. Masterov Feb 27 '15 at 2:51
1
$\begingroup$

It sounds very much like the definition of a non-ergodic process (measures over realizations not being equal to measures over time). Sadly, very few interesting real-world phenomena are ergodic. I guess this could be a case for finer-scale sampling and inference, where certain simplifications might be carried out. I'm thinking for examples of small time- or spatial scales, where chaotic behaviour is not observed so predictors can be linearized. But I'm just rambling here.. I'm afraid I can't help you with specific literature on the topic, either. Sorry :/ But interesting question nonetheless

$\endgroup$
1
$\begingroup$

You partially answer your own question by asking for "longitudinal" changes. Cross-section data are called because they take a snap shot in time, literally a cross-section sliced out of a time-evolving society with its many relationship. Therefore, the best inference you can hope to do is under the assumption that whatever it is you are studying is time-invariant, or at least has concluded its evolution.

On the other, the data you are looking for are longitudinal data or panel data for Economists.

A good reference that explains mostly methods but also highlights two prominent examples from Economics is here. Example 2.1 has company investment rates.

Section 3 is a little more theoretical but carries a lot of insight: a panel data model can be \begin{eqnarray} y_{i,t} = \alpha y_{i,t-1} + x_{i,t} \gamma + \eta_{i} + v_{i,t}. \end{eqnarray}

Now, this type of model can capture state dependence, which is (next to unobserved heterogeneity) a common explanation for why people behave differently. Therefore, if you only observe people traveling at a given point in time, your $\alpha$ will be unidentified, meaning you are not aware on how much their travel yesterday has influenced their decision to travel again.

Now, shut down time dependence for a moment but keep in mind that this equation may likely have been the true model.

In a cross section model now, you would drop the subscript $t$ entirely because you only have data in one period. Therefore, you also have no possibility of accounting for the fact that each individual in your data set may have wildly different $\eta_{i}'s$, which will bias your regressions upward generally, at least when true model is dynamic. This is likely the reason of the overestimate, because of an unobserved individual effect (can be common, too), that you did not measure but that was reflected in your cross-section study.

Now, enter panel data again. What we can do is subtract the mean over time of each variable which, given the mean of $\eta_i$ is constant over time, would eliminate this term. This transformation (others are possible) allows you to focus only on the dynamics (and in fact you'd lose any time-invariant regressors).

Now, this is the main difference between cross-section and panel data. The fact that you can eliminate the time-invariant effect because you have that time variation allows you to remove certain biases that cross-section estimation doesn't allow you to detect. Therefore, before you contemplate a policy change such as a higher tax on traveling because you expect people to travel and you want more government revenue, it is more useful to have seen the phenomenon over a few years so you can be sure that you are not capturing unobserved heterogeneity in your sample which you interpret as a propensity to travel.

To estimate these models, it is best to go through the reference. But beware: different assumptions about people's behavior will make different estimations procedures admissible or not.

I hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.