Regression with 'stacked' data

Question

I've occasionally seen people do something like the following. Let's say we have a survey battery with k questions, for instance an evaluation of red jelly beans, an evaluation of green jelly beans, etc. Then we have another battery of related questions, such as an evaluation of the color red, green, and so forth. The data might look like this:

id	y1	y2	y3	x1	x2	x3
1	10	9	5	6	10	5
2	9	5	10	0	10	5

The data is then 'stacked', so that for each respondent there are k rows:

id	y	x
1	10	6
1	9	10
1	5	5
2	9	0
2	5	10
2	10	5

Finally, a model such as OLS is fitted, with y as the dependent variable and x as the independent variable. Most likely there are several independent variables, and they may have been constructed in more complicated ways.

I'm trying to understand whether this is a sensible thing to do, and what issues might arise. It feels a bit fishy: observations are not independent (there are k observations for each respondent, and n (the number of respondents) observations for each type of jelly bean). A model with mixed effects would seem reasonable, as suggested in answers to similar questions here and here.

If, however, the analysis was performed as described, what should I be cautious about when interpreting the results?

Answer 1

Indeed, as you already point out:

observations are not independent (there are k observations for each respondent, and n (the number of respondents)

The OLS parameter estimates will be unbiased, as long as there is exogeneity ($\mathbb{E}[\epsilon_i|x_i] = 0$). The covariance matrix of the parameters, however, in OLS given by:

$\text{Var}[\hat{\beta}∣X] = \sigma^2(X^\top X)^{−1}$

will over- or underestimate the covariance of the parameter estimates (to estimate the covariance, the unknown quantity $\sigma^2$ is replaced with its estimate $s^2$). Standard errors, which are used for inference (e.g., hypothesis tests), are given by the diagonal of this matrix. For valid inference, the estimated covariance matrix needs to account for the correlated nature of the observations. Zeileis et al (2020) give an extensive overview of such corrections, a.k.a. 'clustered covariances' or 'robust standard errors':

Zeileis, A., Köll, S., & Graham, N. (2020). Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R. Journal of Statistical Software, 95(1), 1–36. https://doi.org/10.18637/jss.v095.i01

Answer 2

The kind of transformation you are talking about is often (e.g. in Stata) called "reshaping" the dataset from a "wide" format to "long" format. It's usually used when (for example) x1, x2, and x3 represent different scores on a particular question by the same PERSON (the person who's id is 1) on three different occasions. In this understanding the first "wide" dataset is "person level" - each row represents an individual, and the second "long" dataset is "occasion level" each row represents an "occasion" but the occasions are "clustered" within people. As you allude to the "clustering" of observations in people violates various assumptions about independence etc., but also provides lots of opportunities for analyses that aren't possible in a wide dataset. Multilevel models (aka HLM, aka "mixed models" aka random effects models, aka 12 other things...) are designed to deal with all of this - but are also used to deal with data where individuals are "clustered" in groups at the same time point (so you have a dataset of students in schools) and some variables (e.g. school size) are at the "cluster level" (reshaping is seldom necessary for these kinds of datasets because the data are almost never in "wide" form to begin with).

However, your example is a little odd, in that there doesn't actually seem to be any longitudinal or multilevel structure to the data: the "x" items are just one bank of questions and the "y" items are another. So I'm not sure it if would make sense to treat this as a multileveled dataset, or even what the benefit would be of doing so. For example, when you reshape from wide to long, you generally create a "time" variable that shows you what "wave" of data collection each row corresponds to (e.g. 1, 2 or 3) ...but that doesn't make sense here, because it doesn't seem like x2 and y2 have anything in common that would cause us to think of them as being at the "same time."

If the data you want to analyze really looks like your example, I don't think a reshape would be a good idea. However, if you constructed your example to try to get a handle on the idea of reshaping in general, I'd look more into multilevel models, especially for longutidinal or panel data, since that's when these sorts of transformations are often used.

Answer 3

As there is no time variable, then it's not a stacked panel data, so those techniques don't apply to the letter.

So, it's a cross sectional data, where you know there are groups of answers (color of jelly bean). You could try to run the stacked data model, with the advantage of higher number of observations, but chances are you find high heterogeneity.

It all depends on what's your goal. Do you want to separate your results depending on the color of the bean.

Assuming you want to stay in OLS:

Option 1: You could try three models with $n=2$ as $y_c = f(x_c)$ for each color ($c$), assuming you have more data, it should be ok. Separate conclusions.

Option 2: You could try the stacked data model with $n=6$, but you'll need to add a "color" variable, to control for that feature. This variable should be coded as a factor, which is quite similar to a fixed effects model. Thus, your model will be $y = f(x,c)$.