1
$\begingroup$

I have some large amount of people take test A. Based on the scores on test A, I assign them either to group A1 or to group A2. A1 are the people with scores at least 50% on test A, and A2 has people with scores less than 50% on test A.

It turns out that half the people are put into group A1 and half the people are put into group A2.

Then, I have people take test B, and do the same thing, grouping into B1 for the high scorers, and B2 for the low-scorers.

Now, the data is analyzed, and what is reported is that "people in group A1 were more likely to be in group B1."

Does this imply that if you're a person who was not in the original study (but you come from the same population as people in the original study), and you take test A and find out you're in group A1, that you're more likely to be in B1 (after taking test B)?

(I'm making the assumption that the findings for the people in the test generalize to the population.)

The curiosity is whether a probability about a group can always be extrapolated to individuals within that group.

EDIT: For context, I'm looking at studies regarding implicit bias. The studies have the aggregate data as to how people who exhibit a strong implicit bias perform on particular tests (such as the first-person-shooter task). I'm curious if this means that if someone takes the first test and has a high bias, if this means that they're more likely to perform on the second task in the way that people similar to them in the first task did.

In short, if someone has an implicit bias, are they more likely to act in particular ways? Or is the effect only at a group level, and I can't tell anything about an individual based on the result from the implicit bias test.

$\endgroup$
  • $\begingroup$ sounds like you want to form "representative groups" that you can predict one person's response based on other people's response. what is the variation in your groups? it may be useful to plot a histogram of scores for each group - do they overlap or separate? $\endgroup$ – probabilityislogic May 2 at 8:10
1
$\begingroup$

That is exactly what an statistical test is for, but, you must make sure that your sample of 30 individuals has the power to test your hypothesis. The best is to perform an a-priori power analysis, before you sample 30 individuals. Otherwise, how did you decided 30 was enough?

It would also depend on the size of the population you are sampling from, if there are 32 individuals total and you sampled 30, the answer most probably will be yes. Still, the correct way is by performing an a-priori power analysis and not assume too much.

To understand biases is a very difficult task and there is always the risk of finding unexpected biases. So, you should think thoroughly in your experiment beforehand. Without knowing more 30 seems like a very small sample size, but it's hard to judge without further knwoledge. If you are sampling a quite rare population from a far away location, you might acknowledge your limitations.

$\endgroup$
  • $\begingroup$ Thank you for your explanation. Since I intend the question to be more general, I've taken out the usage of a particular number of people (60 or 30). $\endgroup$ – Pro Q May 1 at 23:26
1
$\begingroup$

Based on the analysis you did, the claim is indeed about individual behavior. In your example, you are interested in the probability of being in B1 vs. B2 for an individual in A1. For that individual, their outcome of B1 or B2 is governed by a parameter that applies to them (it may apply to others as well, but the focus here is on the individual). You are interested in estimating this parameter. Again, you want to make a claim that says for an individual in A1, they are more likely to be in B1 than had they been in A2. It happens to be that the best way to estimate this parameter is to assume all units in your sample are independent and identically distributed and then estimate the parameter in the sample. It's true your sample estimate is based on aggregating your sample, but the interpretation of the estimated parameter (given the assumptions are correct) refers to each individual.

I'm not sure how deep into your statistics education you are, but a conceptual shift that really solidified this concept for me is the distinction between sampling an individual from a population and sampling a score for an individual from the distribution that governs that individual's possible scores. That is, I personally have a probability distribution that governs my values on some variables, and each observation is a draw from this distribution. Note I don't need to make a statement about what group I'm in or what population I'm a part of to make this claim. It is this latter focus on the individual's probability distribution that makes it clear our inferences (in some cases) are about the data-generating model for an individual's score rather than an aggregate statement about some broader population distribution comprised of many individuals.

That said, there are many instances in which patterns in groups are distinct from patterns that occur at the individual level; Simpson's paradox and the ecological fallacy are perfect examples of this. For example, larger animals tend to have longer lifespans on average, but within a species, larger animals have shorter lifespans (i.e., because of heart problems, etc.). The explanation for this apparent paradox is that species that are larger on average tend to live longer on average, but within each species, smaller members of the species live longer. In this sense, an aggregate finding about a population does not apply to the individuals because of group-level confounding (i.e., average size of a species causes both individual sizes and variation in lifespan). While it's true that larger animals do indeed live longer, this is a marginal claim rather than conditional claim, when a statement that more accurately depicts the relationship within individuals is a conditional claim. In your example, it may be that marginally being in A1 is associated with being in B1, but if you were to perform a conditional analysis, you might find that being in A1 is negatively associated with being in B1 within levels of some other variable C. For example, maybe older people are both more likely to be in A1 and B1, but among people of the same age, there is no association between A and B.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.