I am following this paper: Measuring, Predicting and Visualizing Short-Term Change in Word Representation and Usage in VKontakte Social Network where in Differencing Statistics section they describe that they performed first-order differences of their trends.

More specifically, for word frequecy time series and tf-idf time series they have taken subtraction, and for word embeddings they have used cosine similarity.

My first question is why they have considered first-order difference without doing remaining tragectory clustering starightly using the original time-series data?

My second question is how first order difference is performed using substraction and cosine simialrity as it is not mentioned in the paper?

I am happy to provide more details if needed.


1 Answer 1


Regarding the first question: They want to cluster the time series into the categories: increasing, decreasing and constant. Those categories refer to change... the value of the time-series itself is not important, only the first derivative. That's why they calculate the first derivative.

Regarding the second question: For the scalars $f$ and $ \mathcal{X}$ subtraction is straight forward: $$\Delta \tau_f(w) = \Big(\Delta f_{t_0,t_1}(w),\Delta f_{t_1,t_2}(w)..., \ \Delta f_{T-1,T}(w)\Big)=\Big(f_{t_1}(w)-f_{t_0}(w), \ f_{t_2}(w)-f_{t_1}(w) ..., \ f_{T}(w)-f_{T-1}(w)\Big) $$

For the vector $e$ they use cosine similarity instead of the subtraction: $$\Delta \tau_e(w) = \Big(\Delta e_{t_0,t_1}(w),\Delta e_{t_1,t_2}(w)..., \ \Delta e_{T-1,T}(w)\Big)=\Big(\text{cosim}\left(e_{t_1}(w), e_{t_0}(w)\right), \ \text{cosim}\left(e_{t_2}(w), e_{t_1}(w)\right) ..., $$ $$\ \text{cosim}\left(e_{T}(w), e_{T-1}(w)\right)\Big) $$

How to actually calculate the cosine similarity you can find everywhere.

  • $\begingroup$ thanks a lot for the great answer. Your answer for the second question is very clear to me. However, I am still not clear what happens with first derivative. is it like taking the slope of two time-intervals? Looking forward to hearing from you :) $\endgroup$
    – EmJ
    Aug 19, 2019 at 23:14
  • 1
    $\begingroup$ The derivative is the slope. However, they don't have continuous but discrete data. So they can only approximate the slope by taking finite differences of the subsequent values of the time-series. Imagine you have 4 eggs at t=1 and 6 at t=2. Then your net change over that time is 6-4 =2. That is what they are calculating for every time point. $\endgroup$
    – PascalIv
    Aug 20, 2019 at 13:02
  • $\begingroup$ @Pascallv thank you :) $\endgroup$
    – EmJ
    Aug 25, 2019 at 9:13
  • $\begingroup$ @Pascallv please let me know your thoughts on this question: stats.stackexchange.com/questions/423116/… :) $\endgroup$
    – EmJ
    Aug 25, 2019 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.