1
$\begingroup$

I have a dataset with the following datetime values of part delivery orders:

OrderId RequestedDeliveryDate ActualDeliveryDate OrderCreationDate OrderChangeDate PartReplenishmentTime
1 2001-03-09 2001 -03-09 2001-03-02 2001-03-02 7.0
2 2001-03-12 2001-03-12 2001-03-02 2001-03-02 10.0
... ... ... ... ... ...

PartReplenishmentTime is the timeframe between OrderCreationDate and ActualDeliveryDate.

I have been given the task to "create a neural network with the purpose of finding anomalies in the dataset."

I am unsure about the following things:

  • I know that auto-encoders are a form of NN which can be used to detect anomalies in datasets, but in all examples and papers I found, people were using auto-encoders with labled data, e.g. with images or time series data like ECGs. They trained their model with labeled data and then fed it some unlabled data to determine if it is anomalous or not.
    For my understanding, the given data is unlabled, as I don't have examples of anomalous and right data.
  • I don't know what to do with those arbitrary dates in this context; I thought about transforming them into timeframes like the PartReplenishmentTime, but am unsure if a NN can make sense of it. What can I do with the given data to get good inputs for a model?
  • The more I try to research the subject, the more I doubt if the task given to me is the right choice for the problem at hand. Maybe a linear regression analysis would be more sensible here?

I don't really know where to start developing a model, to be honest.

$\endgroup$

1 Answer 1

0
$\begingroup$

Autoencoders and labels:
Autoencoders are usually used with unlabeled data, so I am a little surprised by the result of your literature research. Actually, autoencoders are one of the standard tools for doing unsupervised anomaly detection (UAD). Maybe there is some misunderstanding: Labeled data in anomaly detection means that the label indicates whether the data point is anomalous or not. In this case, anomaly detection is called supervised anomaly detection and is the "same" as binary classification. Nevertheless, you can of course search for anomalies with UAD in data that consists of features together with some "labels" as long as you consider both as independent attributes.

Your input features (the dates):
I could imagine that a good choice of input features would be the differences between OrderCreationDate and all the other dates.

The right model:
I don't know much about your situation, however, I would probably not start with autoencoders but rather use one of the simpler models, e.g. those provided by sklearn like isolation forest.

$\endgroup$
3
  • $\begingroup$ Thanks for your answer! I also came to the conclusion that the dates as such - without any preprocessing - do not contain relevant data for detecting anomalies, as they don't follow any schedule or regularity, so I think your proposal of taking differences makes sense. I will read into isolation forest and try to find a solution using it. $\endgroup$
    – Velioc
    Commented Jul 10, 2022 at 13:25
  • $\begingroup$ @Henry If the answer helped you, please consider upvoting it. See: What should I do when someone answers my question? $\endgroup$
    – frank
    Commented Jul 10, 2022 at 14:05
  • $\begingroup$ I did, but I don't have enough rep on CrossValidated for it to display $\endgroup$
    – Velioc
    Commented Jul 12, 2022 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.