visualizing repeated measure longitudinal discrete outcome

Question

This may be a repeat of some questions asked here before, I apologize in advance if thats the case. At at rate, here's my question.

I am dealing with a binary outcome, wheezing=Yes or wheezing=No, this outcome wheezing is measured repeatedly for about 40,000 kids. Unevenly evaluated , some kids are evaluated once , some kids are evaluated 163 times, many others in between this range.

My dataset is like this.

    Id    Date        Age     Wheezing_Status
    121   1995-11-23  0.11    No
    121   1997-06-20  1.18    No
    121   2001-07-25  3.12    Yes        
    19    1998-12-20  5.16    No
    17    2002-01-14  1.29    No
    17    2003-11-28  2.67    No        
    17    2007-03-28  4.12    Yes
    17    2012-04-23  11.23   No
    .     .           .       . 
    .     .           .       .
    .     .           .       .
    .     .           .       .
    153   2006-04-21   3.18   No        
    153   2011-01-08   7.13   No
    153   2016-08-30  11.25   No
    119   2003-08-02  23.47   Yes

    

The main factor is, once the kid is wheezing=Yes, the kid should be flagged wheezing=Yes during subsequent visits or evaluations.

I have reasons to believe there is some error in the dataset and there may be some cases or some kids who may have been flagged wheezing=No , AFTER, diagnosed with wheezing=Yes. Example : ID 17

How do I either programmatically or visually identify these cases, kids documented as wheezing=No , AFTER, diagnosed with wheezing=Yes ? Especially when there are repeated measure data for 40,000 kids ?

Answer 1

If you represent the wheezing status with a value that can be ordered (such as a number) whereby Yes is represented as the larger number, and order the data by age (or date), then the correct status is the cumulative maximum of these values. The errors occur where there are discrepancies between the recorded value and this cumulative maximum.

For instance, consider a patient whose recorded status values over time were the sequence

No, No, Yes, Yes, Yes, No, Yes

The cumulative maximum is

No, No, Yes, Yes, Yes, Yes, Yes

(It is No until the first Yes appears, after which it is forever Yes). Upon comparing these two sequences term-by-term, you will identify the error in the sixth term.

If you maintain these data in a data.table object in R (using the data.table add-in), this two-step process requires just two lines: one to make sure the data are sorted by age and one to compare the cumulative max to the data:

setorder(X, Age)
invisible(X[, Error := Recorded != cummax(Recorded), keyby=.(ID)])

It relies on equating TRUE with the number $1$ and FALSE with the number $0.$

There are very many ways to plot this. ggplot2 will produce something like what you describe:

(Click on the image to see a larger version.) The red dots locate the erroneous values.

This plot was produced by generating a dataset of 623,012 observations for 40,000 subjects. I created errors in almost 100 cases (at random) This plot shows all the erroneous data, plus a few randomly chosen subjects without any errors (for comparison).