7
$\begingroup$

I am currently working on the classification of pulmonary diseases using spirometry. This is a procedure in which the patient blows air in a tube and we collect air volume, pressure, etc, in order to obtain the spirometric parameters.

My question is : If I perform the spirometry in the same patient three times, can I consider these three exams, at the same day, to be three different data points in my training of testing set or it is better to average the results and consider only one data point? If the patient comes in a different day, can I considered this exams to be independent?

I think it is okay to consider the exams to be different data points, but I would like to hear other opinions.

$\endgroup$
2
  • 2
    $\begingroup$ FWIW, for spirometry the industry standard is to run each patient 3x. If the measures vary by <10%, it is considered a good test. Most advocate averaging the measurements, but a minority advocate using the best under the theory it's a more accurate measurement of what their lungs can do. From a stats point of view, either is a good measurement as long as the same method is used for everybody, & using the average yields a 58% reduction in measurement error relative to best. There isn't an appreciable trend except for patients w/ COPD b/c the huffing & puffing leads them to clear their airways. $\endgroup$ Mar 6 '14 at 17:37
  • $\begingroup$ @gung: Wow. This is some very impressive and very unexpected knowledge. $\endgroup$
    – amoeba
    Mar 6 '14 at 23:12
11
$\begingroup$

They are definitely three different data points, but they are also definitely not independent (whether they are same day or different day). What you should do about this depends on the goals of your analysis, but it is likely that a multi-level model is a good choice. Averaging the points is also possible, but it reduces variability and eliminates the ability to look at trends over time.

$\endgroup$
0
8
$\begingroup$

I mostly concur with @PeterFlom's answer. In my opinion, you should not average your data (you are basically throwing away 2/3 of your information, why would you want to do that?), but you should definitely account for the fact that measurements on the same patient will tend to be closer together than measurements on different patients. In such a situation, I usually recommend mixed linear models, which are a simple instance of the multi-level models @PeterFlom recommends.

Specifically, you would use a generalized linear mixed model. The link function would be logistic, as in "ordinary" logistic regression. However, the functional form would include multiple observations on each participant, modeled by a random effect, just as in "ordinary" linear mixed models, $y∼F(Xβ+Zγ)$. In R, you can fit this by glmer() in the lme4 package, using the binomial family. For prediction, you could use a single measurement.

Whether or not a mixed model predicts better than a non-mixed model in a particular setting is hard to say, of course. What the mixed model does is account for intra-person variability. If you just average the three original data points, you lose all the variability between measurements, so you will be too optimistic about your ability to predict from a single new observation.

If, on the other hand, you simply throw in all observations without taking the grouping into account, you will again be too optimistic, as all standard errors will shrink. Think of what would happen if you started with a single observation per participant, say 100 data points... and then simply copied each observation 100 times. You would end up with 10,000 "observations" and far smaller standard errors than with the original data, although you didn't enter any new information.

In addition, mixed models allow modeling other grouping factors, like the location, its specific demographics, its staff, diagnostician characteristics etc. So they are a lot more general than averaging.

$\endgroup$
10
  • $\begingroup$ (-1) The OP wrote that his or her goal is "classification of pulmonary diseases using spirometry". Your answer is about modeling spirometry results as DV, but what OP wanted is using spirometry results as IV to classify diseases... $\endgroup$
    – amoeba
    Mar 5 '14 at 22:01
  • 1
    $\begingroup$ @amoeba: I think you may have misunderstood my answer (and possibly the others you downvoted as well). I did not talk about modeling spirometry results as a DV, but as an IV - with the challenge that the measurements are correlated for each participant. My suggestion was that the OP should not average the three measurements and use the average as an IV, but to use all three measurements as an IV and account for the dependence between each participant's multiple measurements using mixed models. Please reconsider your downvote on my and the other answers. $\endgroup$ Mar 6 '14 at 7:21
  • 1
    $\begingroup$ I will be happy to revert my downvote, but let me see if I understand. Let's say that we want to predict single categorical $y$ (healthy/ill) by a set of spirometry observations $\mathbf{X}$ (air volume, pressure, etc). I guess you are talking about a logistic regression model (do you?), $y\sim F(\boldsymbol{\beta} \mathbf{X})$. Now for each participant we have three sets of spirometry measures, $\mathbf{X}^{(i)}$ with $i$ from 1 to 3. How do you include this into the glm? I am not familiar with mixed models for classification, that's why I (probably) got confused. If so, I apologize. $\endgroup$
    – amoeba
    Mar 6 '14 at 10:16
  • $\begingroup$ (cont.) I forgot to add that at test time we want to predict $y$ for a new patient by having only one instance of $\mathbf{X}$, not three. $\endgroup$
    – amoeba
    Mar 6 '14 at 10:31
  • 1
    $\begingroup$ You would use a generalized linear mixed model. The link function would be logistic, as in "ordinary" logistic regression. However, the functional form would include multiple observations on each participant, modeled by a random effect, just as in "ordinary" linear mixed models, $$y\sim F(X\beta + Z\gamma)$$. In R, you can fit this by glmer() in the lme4 package, using the binomial family: cran.rstudio.com/web/packages/lme4/lme4.pdf And yes, for prediction, you could use a single measurement. $\endgroup$ Mar 6 '14 at 11:24
5
$\begingroup$

The three exams are different data points. Though they are clearly not independent (nor random) observations of all possible exams in your population of interest, at least for any analysis I can imagine.

Others have emphasized that you may do well to include those data points in your analysis (since you already have them), as simple replicates within patient [a nested design] or including "time/visit" as an absolute (e.g. date) or relative (number of visit) variable of interest [some form of repeated-measures design], if interesting. I agree that this is the most interesting (and probable) scenario.

However, it may not be necessary, pay for increased complexity, or improve your conclusions if you are only interested in between-subjects variables. Let's say that you only care for differences between males and females, or you want to explain air volume by patient age. Since you know that you can not properly characterize a patient in one blow (cause measurements result variable even for the same patient at the same moment), then you take several measures and average them. You don't care about that variation, it's just inevitable; you just want to get as close as possible to the "true" (mean) value for that patient (at/in that time). This may be the most reasonable analysis.

[Check this paper for a good read about simplicity vs. complexity in statistical analyses.]

$\endgroup$
1
  • $\begingroup$ Thanks for the link to a nice paper, it makes a very good point! $\endgroup$
    – amoeba
    Mar 6 '14 at 14:12
1
$\begingroup$

In accordance with the other answers (no, these observations are certainly not independent, so what do you do about it)....

But do you want to use this information to predict other variables? Many of the suggestions so far seem to be assuming you want to use spirometry as a dependent variable, and thus modelling the error is more straightforward (using a multilevel model). If you instead want to use the spirometry measures as an independent variable, you would be well served by using a confirmatory factor analysis model with the 3 repeat measures modeled as indicators of a single underlying latent variable. The variance of the underlying latent variable is that shared by all three measures, and thus a better reflection of what you are really after (compared to taking the mean, for example).

$\endgroup$
1
  • 1
    $\begingroup$ I am not sure factor analysis would be useful here: I guess OP wanted to use the classifier to classify the disease given a single spirometry measurement of a new patient, without waiting to collect three measurements first (so you would not be able to apply FA on the real test data). $\endgroup$
    – amoeba
    Mar 6 '14 at 14:10
0
$\begingroup$

the measurements can be independent or not. if you describe the measured value as $y_t=x_t+\varepsilon_t$, where $x_t$ - true value, and $\varepsilon_t$ - measurement error, then independence means that $cov(\varepsilon_t,\varepsilon_{t-i})=0$ for all times. this may or may not be true. if you have two measurements one immediately after another then it's most likely not true. if two measurements were time separated but conducted buy the same technician again this may not be true. etc.

on the other hand it must be possible to setup the measurement in a way that $\varepsilon_t$ would be independent of each other and the $x_t$.

$y_t$'s are most definitely not independent through $x_t$ correlations, but that's not what is meant by independence

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.