0
$\begingroup$

Data

I have a measurement with a detection threshold. I can see that one group has more succesfull measurements (larger number) and also have more higher measurements (longer tail). Comparing the mean shows no difference, intuitively I think that is because of the detection threshold.

Because I see more measurements and higher measurements in one group, I think that the mean in that group is really higher. But the real mean is probably below my threshold. Also, the closer a measurement is to the detection threshold, the larger the probability that it is a false positive.

Incidentally, my data is also hierarchical, but this might not be essential to the question. In each of my two groups I have about 20 subjects and for each subject I have a number of measurements N (between 20 and 100). Between subjects, I assume measurements are independent. Within subjects, I assume measurements are dependent. I use nlme in R for my analysis.

Approach

If I take the top n measurements for each subject, there is a significant difference between groups (tested with lsmeans and contrast). I took n to be the smallest number of measurements I had in one subject. So for the subject with the least number of measurements I included all measurements, but for the subject with the largest number of measurements I included only the top n which was about 20%.

dat = loadsomestuff('from here')
fit = lme(measurement ~ group, random = ~1|subject, data = dat)
ctrst = contrast( lsmeans(fit, pairwise ~ group, adjust="tukey"),
                 "trt.vs.ctrl", ref=1)

ctrst 
 contrast    estimate         SE df t.ratio p.value
 B - A    -0.08378559 0.06237481 24  -1.343  0.1918


dat.top20 = dat %>% group_by(subject) %>% top_n(n = 20, wt = measurement)
fit.top20 = lme(measurement ~ group, random = ~1|subject, data = dat.top20)
ctrst.top20  = contrast( lsmeans(fit.top20, pairwise ~ group, adjust="tukey"),
                        "trt.vs.ctrl", ref=1)

ctrst.top20 
 contrast   estimate         SE df t.ratio p.value
 B - A    -0.3088250 0.10896501 34  -2.834  0.0148

I pretty sure this approach is not valid, since if sampling the same distribution twice and taking more samples the second time, the mean of the top n samples would be higher for the second time. I also suspect just testing the difference between the values straight away is not the best approach, as by inspecting the data manually its very suggestive that one group has higher values.

Question

What are ways to compare the values found in these groups? (I assume my approach is invalid)

Progress

I found one similar question that has had a response to it. I dont completely understand it, because I have no experience with survival data and that question posts values like <5 while I have definite values like 5.1.

$\endgroup$
  • $\begingroup$ The definition of censoring in the question you reference is the same as yours. Whether or not your question is about survival, many have found survival analysis techniques to be applicable and useful to your situation: see, e.g., Dennis Helsel's NADA book. You haven't fully disclosed your method, because you haven't explained how many total measurements there are for each subject, nor indicated how they might be dependent, nor said how you assess the significance of any difference you might find. There are many alternatives: see nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=P10055GQ.TXT $\endgroup$ – whuber Aug 8 '17 at 14:38
  • $\begingroup$ @whuber Thanks, I will have a look at that link. I added some of the information you were missing. However, my original intention was to just ask the general question is it valid to take the top n measurements per group and compare their mean? , not regarding my exact analysis with nlme in R and the hierarchical nature of my data. So my actual study is more involved than that one question, but I think that might make the question less clear and has no influence on the answer. What do you think? $\endgroup$ – Leo Aug 8 '17 at 15:07
  • $\begingroup$ The answer to "is it valid" depends on the details of the data and on how you go about comparing those top-$n$ measurements. There are useful versions of this approach based on Maximum Likelihood, nonparametric tests, survival analysis, and other procedures. $\endgroup$ – whuber Aug 8 '17 at 15:35
  • 1
    $\begingroup$ Have a google for "trimmed means" (e.g. dx.doi.org/10.1002/pst.1768, where the approach is used for totally different reasons). However, if you are willing to assume a parametric distribution (whether normal, log-normal, whatever), treating the data as censored is much more efficient. $\endgroup$ – Björn Aug 8 '17 at 15:46
  • $\begingroup$ From your description, I'm wondering if you don't want to look at the mean at all, but instead want to look at a quantile, like the 75th percentile, say. For two groups, you could accomplish this with permutation test, or for a more complicated model, use quantile regression. $\endgroup$ – Sal Mangiafico Aug 9 '17 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.