0
$\begingroup$

I am analyzing (extreme value analysis) the dataset which contain daily rainfall over 100 years of a single location. However there are around 500 missing values on the whole dataset. In this case the exact reason why data is missing is not known, but it is highly likely that it is due to flood. The place where rainfall is gathered is sorrounded by flood. That means there is high probability of that data might contain high rain fall values(Missing Not at Random). So mean substitution or omitting missing values won't be a good choice.

What are the options that are available in this case ( I am open to classic statical approaches as well as machine learning based approaches as well).

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

This might not be the best or most scientific approach, but since your observations probably have an inherent order (a date, perhaps), - maybe you can use a windowed function to look pre and post each missing observation, and then impute the missing value with the mean. To take it even further, if you could regress the pre and post values and predict the missing value, but with extreme values this might not be very accurate.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Mean imputation will grossly bias the flood estimates (low). A regression will depend entirely on the model chosen, essentially papering over the problem of the missing data. Flood events--like many other extremes--bear almost no relationship to events observed the remaining 99% of the time, yet account for a great deal of the total flow (and other hydraulic actions) in any river. Thus, without a scientific flow model or at least some reliable data obtained during representative flood events, the imputation task appears to be utterly hopeless. $\endgroup$
    – whuber ♦
    Commented Sep 19, 2014 at 20:30
  • $\begingroup$ For the dataset contribution of extreme values in the monsoon season is quite high. the assumption/ observation as per whuber "bear almost no relationship to events observed the remaining 99%" might not be true for all the seasons. $\endgroup$
    – carl
    Commented Sep 29, 2014 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.