1
$\begingroup$

How should missing observations be handled to create ACF-Plots?

Let's assume we have a time series t = [1, 1, 1, 2, NaN, 3, 2, NaN, ...].

What schould be done with these missing data points? I have not found any function for python that can handle missing data points.

$\endgroup$
0
$\begingroup$

Imputation data is a large area of ​​research and complications. One must be very careful.

In this specific case, you need to understand if:

1) Are the missing values ​​random? If so, we can try some basic predictive model (such as a non-temporal regression) to try to fill in the null values. If they are not random, as is most common, you must understand the fact that the missing is generated (such as replacing zero values ​​with the absence of information that obviously has different meanings)

2) Are the intervals between missing values ​​regular? If they are regular you will have a "seasonal effect" in your autocorrelated series (either in time, space or other dimension) and this will make it difficult to fill in, perhaps an alternative in this case is a interpolation of non-missing results. If this range is extremely rigid (for example, you never have data on Fridays), one alternative is to drop every Friday from the model and predict without that data (taking care not to skew the prediction)

An alternative approach would be to use machine learning models that treat autocorrelated data and pass a "warning" variable that in that field value is missing, for example an LSTM. With the disadvantage that you will lose interpretability of the model (as a SARIMA would do)

$\endgroup$
  • $\begingroup$ Just a note that you have to be careful with your wording here. When you say "Are the missing values random", I think you really mean to say are the values missing completely at random. It is a small distinction but an important one. There are three cases of missingness. 1. Data is missing completely at random (missing is entirely random and not dependent on variables observed or unobserved). 2. Data is missing at random (missingness is dependent on observed variables). 3. Data is missing not at random (missingness is dependent on unobserved variables). $\endgroup$ – astel Jan 21 at 17:01
  • $\begingroup$ Astel, I do not know if I understand your point. Given a random missing, the fact that it is random is already implicit the exogeneity of the event, so its absence does not depend on observed or unobserved variables. $\endgroup$ – sn3fru Jan 21 at 17:16
  • $\begingroup$ My point is that when you say "Are the missing values random" you have to be clearer. This is because how missingness is typically defined. As I mentioned above, missing at random and missing completely at random are not the same things. So when you say, are the missing values random, what I think you mean is are the missing values missing completely at random (ie. does not depend on observed or unobserved variables) but what it looks like you are saying is are the values missing at random (ie. missingness depends on observed variables). $\endgroup$ – astel Jan 21 at 22:31
  • $\begingroup$ I had not thought of that, but I think you're right. With the intention of making the text clearer, could you suggest a correction by editing my text? Thank's for your time. $\endgroup$ – sn3fru Jan 22 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.