We have entities where every entity has start ($s_i$) and end ($e_i$) times and count $c_i$. An entity is important if its interval ($e_i - s_i$) is large and if its $c_i$ is large.

Here's what I did:

In a time window $t$, we take the set of entities that are active (their interval intersect with our window), $entities_t$

Let's define:

$$S_t = \min_{i\in entities_t} s_i \\ E_t = \max_{i\in entities_t} e_i$$

So at time $t$ we can define the features:

$$i\in entities_t: a_{it} = \frac{e_i - s_i}{E_t - S_t}$$

Similarly, we can define:

$$C_t = \max_{i \in entities_t} c_i$$

and the corresponding features:

$$i\in entities_t: b_{it} = \frac{c_i}{C_t}$$

What I'd like to achieve:
I need to devise a formula which need to resemble "how many" important entities are within the time window.

  • Are $a$ and $b$ good as features? If so, how to use them together?
  • if not, how would you define them?
  • I currently don't have quantitive way or some test to asses it, but the $a$ feature is "stronger" than the $b$ feature (In words, the time feature is stronger than the count).
  • Given that, I'd probably need some hyper parameter to weight the two features, right?

Let's begin by examining $a$ and $b$. $a$ is really a proportion of time, which runs from $0$ to $1$. If $a_{it} = 1$, then you have 1 (presumably important) entity in your time window. Likewise if $a_{it} = 0$, then presumably that particular entity is somewhat unimportant.

Similarly, $b$ is also a proportion running from $0$ to $1$, with a value of $1$ indicating one dominating (presumably important) entity, and $0$ indicating a presumably unimportant entity.

You can also assign a weight to $a$ and $b$ to adjust for the fact that $a$ matters more than $b$. Let's call those values $w_a$ and $w_b$. Let's make $w$ also between 0 and 1, so we can keep the notation nice.

Now, since $a$ and $b$ have the same support set, we can take a weighted average to get an measure of importance, let's call it $I$. Then

$I_{it} = w_a*a_{it} + w_b*b_{it}$

With $I$ running from $0$ to $1$. For each entity, if $I_{it}$ exceeds some predetermined value, then it is important.

Each timestamp would be assigned a count of how many important values it has.

There is one immediate danger that I can see

A time stamp with 1 important event either has one important event and a bunch of miniscule events, or 1 important event and no other events at all. Therefore, it may be worth keeping track of the ratio of important events to all events within a timeframe, so you have an idea which case you're in.

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.