# Can fixed effects models be validated with out-of-sample data?

Suppose I have a 2-way unit and time fixed effects model:

$$y_{it} = a_i + b_t +X_{it}\beta + \epsilon$$

I collect data on a set of units $$A$$ and $$B$$, where $$A$$ and $$B$$ are disjoint. The data was collected during the same time. I would like to measure how well the model fit on $$A$$ predicts the within variation in $$B$$. Is it possible to do that?

My thinking is that between unit variation will be different between $$A$$ and $$B$$, since the units are different, but the within predictions made by the fit on $$A$$ should be informative of within variation in $$B$$.

• What is $t$ index referring to Commented Jan 20, 2022 at 22:35
• It refers to time. Commented Jan 20, 2022 at 23:10
• "My thinking is that between unit variation will be different between A and B, since the units are different, but the within predictions made by the fit on A should be informative of within variation in B." How are we supposed to answer the question about the relationships between A and B when we have no idea what these sets of units mean and how they relate to each other? (and I also wonder what the word/term 'unit' is supposed to mean) Commented Jan 21, 2022 at 14:31
• "I would like to measure how well the model fit on A predicts the within variation in B..." You can 1 make predictions, 2 compute the observed variation 3 compute the difference between prediction and observation 4 use whatever statistic you like to express the difference. Commented Jan 21, 2022 at 14:35
• I'm not sure how the nature of units matters. I've always seen fixed effects models being described with a generic meaning of unit. In this case you can consider a unit to be a person. Commented Jan 21, 2022 at 19:32