6
$\begingroup$

Let's say that I want to measure the effect of a treatment on the performance of a firm. However my sample is very small. Let's say 10 firms. It is not possible to observe more firms. All these firms currently have one performance indicator.

Now, what happens if I start measuring 5 performance indicators per firm instead of the one?

One could argue that measuring 5 separate performance indicators, would give more certainty on the performance of the firm and therefore improve the amount of observations to n=50.

The counter argument would be that you are still measuring the same 10 firms.

Maybe the truth is somewhere in between?

I guess it also relates to clustering, for which I added an interesting article (which argues that researchers "over-cluster"):

When Should You Adjust Standard Errors for Clustering?

EDIT:

From the comments to the answer by Dave:

The argument would for example be that profit is a more short term indicator of performance and number of new hires is a more long term indicator. Together they would then give a better indication of the performance of a firm (there is more information).

Now I am wondering if this is perhaps a similar mechanism as measurement error: One measure is not a good measure of true performance (so there is measurement error), but multiple measures could solve this error.

$\endgroup$
3
  • 1
    $\begingroup$ Although the firms might legitimately form a complete "population," this doesn't sound like a relevant sense of the term. Wouldn't your population be something like the hypothetical, and infinite, collection of all possible (treatment, response) pairs? If not, then what's the point of applying any treatment in the first place? $\endgroup$
    – whuber
    Apr 20, 2022 at 11:35
  • 1
    $\begingroup$ @whuber Thank you for your comment, I was attempting to simplify the question. But apparently it was just that that lead to confusion. I removed that part of the question. $\endgroup$
    – Tom
    Apr 20, 2022 at 11:40
  • $\begingroup$ Are the treatments changing if you were to measure more performance indicators on the firms? If so, perhaps you could use a panel data model. $\endgroup$
    – Adam
    Apr 20, 2022 at 14:50

1 Answer 1

2
$\begingroup$

I would argue that you still have $n=10$.

When you measure one indicator per firm, you have ten univariate observations. When you measure five indicators per firm, you have ten multivariate observations.

Either way, $n=10$.

$\endgroup$
3
  • $\begingroup$ Thank you for your answer Dave. But should, if I pool those 5 indicators together, that pooled measurement not lead to an estimation that has more certainty, keeping everything else equal? $\endgroup$
    – Tom
    Apr 20, 2022 at 15:51
  • $\begingroup$ @Tom Do you have an example where you think that works? An example that comes to mind to me is that you measure performance by measuring equity in dollars and also number of new hires. I would not want to pool those. $\endgroup$
    – Dave
    Apr 20, 2022 at 15:56
  • $\begingroup$ It would be an example similar to the one you gave.. Where the argument would for example be that profit is a more short term indicator of performance and number of new hires is a more long term indicator. Together they would then give a better indication of the performance of a firm (there is more information). Now I am wondering if this is perhaps a similar mechanism as measurement error: One measure is not a good measure of true performance, but multiple measures would be.. (Just something that popped into my mind) $\endgroup$
    – Tom
    Apr 20, 2022 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.