2
$\begingroup$

I am comparing energy expenditure outputs from two devices that track physical activity in a free-living setting over 10 days. Doubly labeled water is the accepted criterion measurement for this, but 1) This technique only gives total energy expenditure and 2) I did not use this in my study...

Both of the devices I used have previously been validated against DLW. These devices give a lot more information than DLW, namely time spent at specific PA thresholds and number of 'bouts' at each threshold.

I would like to assess the level of agreement between the two devices. I know that Bland-Altman plots are commonly used to assess agreement between measuring devices but I am struggling to decide which test to use without a reference/criterion measurement. I would guess that a basic correlation would provide some information but was wondering if anybody could offer some more information and/or guidance for a more sophisticated statistical anaylsis. Maybe a regression / sum of squares would be appropriated?

Any help would be much appreciated.

Improve this question
$\endgroup$
3
  • $\begingroup$ I think the Bland Altman plot and correlation are just what you want. You may also want to look at any cases where the differences are very large, to see if you can explain why they are large. $\endgroup$
    – Peter Flom
    Dec 21, 2013 at 13:07
  • $\begingroup$ Thanks. I assumed you needed the 'true'/reference value in order to use Bland-Altman plots. I could perform multiple Bland-Altman plots for time spent in each specific PA threshold which would also help to spot outliers. So a correlation gives info on the validity whereas Bland-Altman gives information on agreement between the devices? As in it would be appropriate to include BOTH in my results? $\endgroup$
    – user36495
    Dec 21, 2013 at 13:32
  • $\begingroup$ Perhaps the exact definition of a B-A plot requires this, I am not sure. But the equivalent Tukey mean difference plot does not (they look identical). Correlation will tell you how close agreement is; plots will tell you more about the disagreement. $\endgroup$
    – Peter Flom
    Dec 21, 2013 at 15:28

1 Answer 1

Reset to default
3
$\begingroup$

1) You can have measures with high correlations and poor agreement. This is explained in the short paper outlining the Bland-Altman plots (which yes is the same as the Tukey mean difference plot).

2) You don't need a gold standard. You just need to make sure you are comparing like with like. Minutes in a given PA category is fine. You could likely even compare the raw output.

3) The hard part is sorting out where any differences arrise. Is it the raw output from the device? Do users wear them differently? Are the cutpoints different?

4) I know there are papers published comparing diffrent models of accelerometer. Perhaps see what they do. Example

5) It is also worth noting that validity is assessed by comparison with a gold-standard, regardless of the method of comparison.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for helping to clarify. Raw outputs may be too difficult to compare due to the sheer volume of data (100+ participants, tracked every min for 10 days). Users wear the devices differently and they measure alternate criteria (both contain an accelerometer). I'm weary of performing a correlation for the aforementioned reasons....maybe I could do one for PAL (i.e. total energy expenditure/basal metabolic rate) and then use Bland-Altman plots for the more in-depth 'bout intensity' analysis. I will continue looking for a paper that compares two devices in a free-living setting without DLW $\endgroup$
    – user36495
    Dec 22, 2013 at 11:56
  • $\begingroup$ Yeah, those raw data outputs are properly big. BUT if you have access to a person/equipment to process it down to something manageable, it could be worth it. $\endgroup$
    – D L Dahly
    Dec 22, 2013 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.