4
$\begingroup$

Pretty much throughout my undergrad and postgrad, I have always learned statistical models predicated upon things like large subject size. I also know that a lot of repeated measures designs typically work at reducing the error variance of a predictor by removing the error in between time periods. I know that this can sometimes be used as a way to increase the power of a test where there are far less participants in the study.

So to ask what is probably a really stupid question, is there a valid statistical test in which you could use a single participant measured hundreds of times to measure an independent and a dependent variable? I would think that normally this would be an issue in terms of generalizability to the rest of the population, but I'm just curious if there is such a thing in statistics, because I couldn't find something like this when I looked online and I imagine there is good reason for it.

$\endgroup$

5 Answers 5

5
$\begingroup$

Single subject analyses are used in some fields, for instance in psychophysics, where it a single participant might make thousands of perceptual decisions (possibly after having an electrode implanted, if the participant happens to be a rodent or money).

The biggest non-technical issue with this approach is that there is no way of knowing whether results will generalise to other people/animals, which is why it's a good idea to use a sample of participants.

$\endgroup$
3
$\begingroup$

Technically, it depends on the assumptions you are willing to make. You could assume that "the rest of the population" behaves exactly like the one participant that you have, and this then allows generalisation. This, however, is an assumption that will be hard to justify, and by definition you cannot check it using data from other participants (as you don't have them). So normally, in a real situation, this should not be accepted.

What you can use is all the machinery of time series analysis (also depending on certain assumptions that may or may not be justified) that will allow you to predict future observations for the same individual (involving tests for things such as the absence of a trend or seasonal variation). For example https://cran.r-project.org/web/packages/funtimes/vignettes/trendtests.html

$\endgroup$
2
  • $\begingroup$ This is actually why I ask. Im collecting data on my own productivity over time. After 6 months ive certainly seen some trends but its probably a bit pseudoscientific to draw any really solid conclusions. $\endgroup$ Feb 12, 2022 at 0:32
  • 1
    $\begingroup$ @ShawnHemelstrand I wouldn't worry much about doing analyses for yourself regarding trend for example. It's up to you how seriously you take the results. The problem with analysing data about yourself is that strange effects may happen through your own awareness of your past data and how this may influence you. Anyway I added a link to trend tests to my answer. $\endgroup$ Feb 12, 2022 at 1:18
2
$\begingroup$

There are statistical methods for hierarchical or longitudinal data, where for example you have a multiple measures on each individual. These methods recognize that multiple measurements on an individual can't be treated the same way as single measurements on many individuals because they aren't independent observations. If you had 100 blood pressure measurements on 2 subjects, treating them as independent observations with ordinal least squares would produce artificially low estimates of standard error. Despite 200 data points, you actually know very little about the overall variance of blood pressure over the population.

The methods are called mixed models, hierarchical models, or random effects models.
https://online.stat.psu.edu/stat510/lesson/10

$\endgroup$
2
$\begingroup$

Addendum to Original Question

I was at the beginning of my statistical knowledge when I first posed this question and since have been exposed to quite a variety of topics. Revisiting this question, I felt the answers here were good, but wanted to add that there is an entire paradigm related to this called n-of-1 designs. As mentioned by others, this involves measuring the same person or a small number of persons several times versus what is typically done by measuring a large sample once or a selected number of time points.

What is quite fascinating about this in relation to my own work is that psychology was originally built off these types of experiments. B.F. Skinner, for example, built the entire theoretical underpinnings of operant conditioning based on the many observations of individual rats. Skinner once noted that:

“It is more useful to study one animal for 1000 hours than to study 1000 animals for one hour.”

The merits of that statement are certainly debatable. In any case, that spirit of experimentation died with the adoption of large sample testing in the 1960s in lieu of NHST-focused results. What that means from an inferential standpoint certainly changes as noted by some already here. The notion of a p-value, for example, changes when we are focusing on an individual, as a sample from a population is supposed to infer upon a population as a whole, whereas if we are only concerned with an individual, the "population" here is the person alone. Thus a p-value means different things here contingent on what you attempt to extrapolate from it.

Some may wonder what the utility of studying one person may be. From a research standpoint, rare populations are already difficult to study (such as those with brain injury), and so applying an n-of-1 design provides a wealth of information that would otherwise not be available for understanding these people. From a practical standpoint, it can greatly inform us of a person's behavior in an applied setting. For example, if a teacher wants to know if a given reading intervention works on a single student who has struggled in her classroom, she can apply an n-of-1 approach and uncover a wealth of information on the actual effect this intervention has over many time points.

Reading through this topic has been interesting, and so I have included some relevant works at the end of this answer for anyone who chances across this question and is interested in learning more someday.

Relevant Works

  • Davidson, K. W., Silverstein, M., Cheung, K., Paluch, R. A., & Epstein, L. H. (2021). Experimental designs to optimize treatments for individuals: Personalized n-of-1 trials. JAMA Pediatrics, 175(4), 404. https://doi.org/10.1001/jamapediatrics.2020.5801
  • Goyal, P., Safford, M. M., Hilmer, S. N., Steinman, M. A., Matlock, D. D., Maurer, M. S., Lachs, M. S., & Kronish, I. M. (2022). N‐of‐1 trials to facilitate evidence‐based deprescribing: Rationale and case study. British Journal of Clinical Pharmacology, 88(10), 4460–4473. https://doi.org/10.1111/bcp.15442
  • Guyatt, G., Sackett, D., Taylor, D. W., Ghong, J., Roberts, R., & Pugsley, S. (1986). Determining optimal therapy: Randomized trials in individual patients. New England Journal of Medicine, 314(14), 889–892. https://doi.org/10.1056/NEJM198604033141406
  • Smith, P. L., & Little, D. R. (2018). Small is beautiful: In defense of the small-N design. Psychonomic Bulletin & Review, 25(6), 2083–2101. https://doi.org/10.3758/s13423-018-1451-8
$\endgroup$
1
$\begingroup$

Two simple examples :

  1. A thousand scientists measure the height of one person a thousand times (one time / scientist), each with their own ruler and methodology. Now we know the height of this person veeeery well. The "true" average height of this person can be estimated with very low uncertainty. However, that does not gives us a lot of information about the "true" height mean of all the population this person is supposed to be from. Confidence interval can not even be estimated in this case. Hence the necessity, if one wants to be able to generalize, to get a sample of many randomly chosen people from this population. Better 1 measurement on 1000 individuals than 1000 measurements on 1 individual.

  2. Suppose I am interested in atmospheric CO2 concentration. There is only one planet so asking what is the true mean CO2 concentration in the population of atmospheres is silly. However we really wish, in this case, to know better of our one and unique atmosphere. So it is appropriate to measure 1000 times the concentration of CO2 in the same atmosphere. Although it would be wise to make each measurement in very different conditions so as to get a global perspective on this unique "individual". Or maybe the question asked is about the evolution of atmospheric CO2 content over time. We would then gain more knowledge by sampling the "same" volume of free atmosphere repeatedly over time.

Take home message : it all depends on the question that is asked. If you have one individual only, don't make hypothesis about the population. But you can learn a lot about this individual ! And maybe add external knowledge / logic to argue in favour of the generalizability of this result.

Asking the wrong question will almost inevitably lead to false degree of freedom => lower uncertainty estimate / anomalous p-values => false claims. That problem of asking the wrong level of question is called "pseudoreplication" and participate significantly in the reproducibility crisis of science in general...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.