1
$\begingroup$

I'm tasked with investigating whether machine learning algorithms can be used to efficiently identify if a certain type of anomaly is present in the temporal spacing of incoming network packets. Identifying the anomaly using threshold tests takes a fairly long time, since it is pretty small. However my initial experiments with ML-algorithms didn't show a lot of promise in reducing this detection time, so I'm trying to mathematically determine wether ML would even garner a significant performance increase.

The problem can be modeled as follows:

Let R and c be random variables from $\mathbb{R}^+$ with known probability distributions and c << R. Of a third random variable X it is known that either X = R or X = R + c.

How many times does X need to be measured to determine within reasonable certainty whether X = R or X = R + c? Under which circumstances do decision algorithms with higher efficiency than simple threshold-tests exist?

The second question is the important one.

This sounds like the kind of problem where a lot of research in its general direction has already been done. Could you give me some recommended reading or refer me to theorems that deal with this type of problem?

My mathematical intuition would suggest that if R and c are normal, threshold-tests are always the most efficient decision algorithms. However it is also fairly easy to come up with situations where this isn't the case (e.g. if the probability density of R has sharp peaks, detecting their movement could be done with higher precision than for the whole of R)

Improve this question
$\endgroup$
17
  • 1
    $\begingroup$ Need more information about the 'certain type' of temporal anomaly in order to even understand what is being asked. $\endgroup$
    – Carl
    Nov 23, 2021 at 2:37
  • $\begingroup$ @Carl Isn't the explicit model good enough? $\endgroup$
    – whuber ♦
    Nov 23, 2021 at 16:32
  • $\begingroup$ @whuber Yes, it is, but only if it is relevant, and we don't know that, do we? $\endgroup$
    – Carl
    Nov 23, 2021 at 19:40
  • $\begingroup$ @Carl Unless you see some kind of contradiction or inconsistency in the question, it is not constructive (and, imho, disrespectful) to suppose that a key clarifying passage might be irrelevant. $\endgroup$
    – whuber ♦
    Nov 23, 2021 at 19:53
  • 1
    $\begingroup$ As for the details of R and c: R is arbitrary network traffic, determining its characteristics will be part of my thesis' literature review, determining c will be part of the project, but it seems to be almost normal $\endgroup$
    – Jeff S
    Nov 26, 2021 at 21:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.