1
$\begingroup$

I'm trying to analyze some fairly sparse data on a recurrent medical symptom, and I don't know what to do with two entries where my data is incomplete.

My overall goal is a bit vague: it's to find a pattern that hopefully will, with the help of doctors, find a cause. The symptom is not very serious, but annoying. Assume full access to all medical records.

I have data going back three years specifying what day the symptom occurred, and which days it did not. However, for two of the events, I only know that it happened "that month".

Example:

2015,4,1,0,
2015,4,2,0,
2015,4,3,0,
2015,4,4,1,comment
2015,4,5,0,
...

(where the columns are year, month, day, 1 if symptom; 0 otherwise, and a comment)

My two incomplete entries look like:

2015,5,,1,symptom occurred twice this month
2015,5,,1,symptom occurred twice this month

Therefore, if I am going to perform an analysis using logistic regression or other methods, like just looking at graphs, I have a problem with these two entries because:

  1. I know the symptom occurred twice on a certain month;
  2. I do not know which day it occurred. So if I guess, or randomize the day, or use an average value, I am concerned I will falsify the data.

How should I treat these two missing "day" values knowing that I otherwise have a complete dataset going back three years?

$\endgroup$
5
  • 8
    $\begingroup$ Your second choice (randomly assigning a day from a uniform distribution) is a valid one. You can repeat this process a large number of times (say 1000) and re-analyse the data each time. By aggregating your results over all 1000 analyses, you should capture the uncertainty introduced by the randomisation. $\endgroup$ – mkt - Reinstate Monica Jul 15 '17 at 14:47
  • 3
    $\begingroup$ Alternatively, you could just choose to analyse your data on a monthly basis and not a daily one. Average other independent variables within each month, and use 'number of events per month' for this variable you are concerned about. $\endgroup$ – mkt - Reinstate Monica Jul 15 '17 at 14:48
  • 2
    $\begingroup$ The best choice of method for dealing with missing data depends on why they're missing in the first place. Is there a particular reason why these two data points are like this, or is it random? For example, if the data is incomplete because some patients had the symptoms and thus were unable to report their condition, then the data isn't missing at random. If, on the other hand, a computer glitch resulted in loss of data, it would be reasonable to assume that these two data points are completely random. $\endgroup$ – LmnICE Jul 18 '17 at 0:55
  • 2
    $\begingroup$ @mkt +1 I agree with your comment, I believe you should post it as an answer. The only thing I'd want to add is the following: in this specific case of two missing values where each can only take one of thirty values (the amount of days in May) you only need to create $30*30=900$ datasets, one for each possible way of combining the two replacement values. Of course, your random draw would approximate this 'permutation strategy' anyway. Additionally, you might lower the amount of required datasets by using multiple imputation techniques and using information from other variables. $\endgroup$ – IWS Jul 19 '17 at 13:24
  • $\begingroup$ @IWS Thanks for the feedback and the useful ideas. I'll turn that into an answer soon. $\endgroup$ – mkt - Reinstate Monica Jul 19 '17 at 15:22
2
+50
$\begingroup$

Turning my comments above into an answer, as per @IWS' suggestion.

Approach 1

Your point #2 above actually describes what I think is the best approach: randomly drawing days from a uniform distribution and assigning them to the observations. This process does involve inventing data, so your concerns are entirely valid.

However, you can accurately account for the uncertainty introduced by this process of data invention through bootstrapping. You can repeat the random assignment (of observations to day numbers) a large number of times, and re-analyse the data each time. By aggregating your results over all analyses, you can capture the uncertainty introduced by the randomisation.

Since you have only two points that are missing dates and both are from the same month, @IWS correctly points out that one can exhaust all possible combinations with 900 datasets (or 870, if you can be sure that both did not occur on the same day). In case you had missing observations across multiple months, correlations between months would have to be accounted for in the bootstrapping process. The true set of possibilities would be much larger in this case. You could choose to analyse all possible combinations or a large but random subset, which should give you basically the same answer. Whether you choose all possibilities or say 10000 possible combinations will depend on the complexity of your models, and the details of your dataset.

Approach 2

Depending on your specific goal, you can also choose to analyse your data on a monthly basis instead of a daily one. In this simpler approach, you could average all independent variables within each month, and use these to predict (for example) the number of times the symptom was observed in that month.

Given that you have just 2 cases of this in the entire dataset, I would say this is not worth it.

Approach 3

As @Semoi suggests, you could also drop these observations. I would strongly advise against this, because you would be throwing away good data that can be accounted for easily using Approach #1. If these points are random ones, dropping may not be a major issue. But if there is anything otherwise distinctive about them - such as them being the only cases where two observations occurred in the same month - you would be biasing your results.

Approach 4 [probably not relevant in your specific example]

You could also use a multiple imputation approach, which uses correlations between variables to infer the values of missing data points. This allows you to generate a number of imputed datasets, each containing a value randomly drawn from the inferred distribution of imputed values. You could then analyse all imputed datasets (similar to approach 1) and aggregate your results across them.

This approach will probably not help in your case because the date a symptom was observed is probably not strongly correlated strongly with other variables in your dataset.

$\endgroup$
4
  • $\begingroup$ I'm glad to see this because I liked your comment. But one question: why in Approach #1 do I need to permute two months, not a single month? The two observations are missing for a single month, after all... $\endgroup$ – martin jakubik Jul 21 '17 at 16:01
  • $\begingroup$ @martinjakubik My mistake, I thought that you had 2 separate months and each of them had 2 observations with unknown dates. But instead, are you saying that you have only 2 observations with unknown dates and both are from the same month? If so, I can edit my answer to address this. $\endgroup$ – mkt - Reinstate Monica Jul 21 '17 at 16:50
  • $\begingroup$ yes two observations with unknown days, but both from the same month. In that case I'm going to pick your answer now, and assume you'll make the correction. Thanks! $\endgroup$ – martin jakubik Jul 22 '17 at 6:13
  • $\begingroup$ @martinjakubik Happy to help! I've now edited my answer to reflect this. I've also added a fourth approach for completeness, but it probably does not apply to your specific problem for reasons I explain above. $\endgroup$ – mkt - Reinstate Monica Jul 22 '17 at 16:28
2
$\begingroup$

If you have plenty days where the symptoms occurred, you probably won't induce a serious error by either dropping the two entries or by including them. Hence, a very simple cross check is to do both once: If the two results differ significantly, you should use a proper statistical method -- see the comments above. Else, you can just describe your cross check and that the result changes only at the xxx% level.

$\endgroup$
2
  • $\begingroup$ Hi, I'll probably accept this answer. I did say that my data is fairly sparse though. So I feel a bit tied down to make as much use of the data as I can. (One thing I just thought of right now would be to make the entire month 2015,5 as a "maybe"... ie., I don't know... every day that month makes a 1/30th contribution to a "positive".) :) $\endgroup$ – martin jakubik Jul 18 '17 at 7:57
  • $\begingroup$ As a physicist, I always try the simple solution to get a feeling of the error bar. Only if I conclude that the simple solution is misleading, I go for the proper solution. Therefore, if you have the time, why don't you use the resampling method described above by mkt. It's a proper method. However, if I would read the comment "occurred twice this month" in a report, I would be doubtful whether or not it occurred at all. So if my result would strongly depend on questionable events, I would not (!) trust the result. I would not risk making an error of first kind without high success rate. $\endgroup$ – Semoi Jul 18 '17 at 19:08
0
$\begingroup$

I would suggest to treat your data as unsupervised data and do clustering to see if you can find any pattern between day and the other features and if there is some pattern,then you can impute the value of day for those two missing entries. Else, as mentioned above if you have a large dataset then I guess it would be fine to drop these two incomplete entries.

$\endgroup$
0
$\begingroup$

Try to derive different features in a way that you can use the available data, for example:

  • Aggregate the dataset by counting monthly occurrences and perform the analysis.
  • Assuming that you have access to medical many medical records, try to relate the symptoms from the known cases with another variable in order to estimate the date on which the missing values occurred.

After deriving some features you can compare the analysis dropping the originally missing cases to the ones with the whole dataset and see whether there are significan differences between them.

$\endgroup$
1
  • $\begingroup$ A variant of this can be to use former data not used in the present data to make a prior distribution, and then using bayesian inference. $\endgroup$ – kjetil b halvorsen Jul 21 '17 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.