PROBLEM: We want to test the impact of an intervention from a public institution on Twitter. The intervention during a week-period of posting certain tweets promoting no toxic language in social media.

DATA: Our data contains the timeline (ten days before and after the first retweet of a tweet from this intervention) of all users that retweeted one of the tweets posted by the public institution. So far we classified the words that their tweets contained in 70 different categories. Besides, we collected general conversation on twitter in case we need to cancel out seasonal trends like san Valentine's day or something like this.

WANT TO KNOW: We need to identify those users that changed the content in their tweets after the intervention date. Firstly I run a t-test to test differences between the mean value of each category after and before the intervention. However, this approach seems to me very rudimental and losing information. I would like to consider this problem as an interrupted time series experiment design.

I am not an expert in time series. So far, I consider applying Causal Impact R package by each user whether or not there were changes in the way they tweet.

Any help would be appreciated!


1 Answer 1


I think an improvement over what you have done is to compute for each user the vector of differences of word usage in the 70 categories, before and after the intervention. Thus, each user would be characterised by a vector of dimension 70. Next, I would perform a cluster analysis to show if some users separate from others in a consistent way; the results of that analysis will suggest, I expect, ways to proceed.

You have to decide what distance to use between the 70-dimensional vectors, as I suspect some word catgegories may be much more populated than others. But surely you can come out with something reasonable.

  • $\begingroup$ Thanks for the answer! That was my first approach, in fact, I used the difference between means value in the t-test to do this. But this approach fails because of the outliers and it is at some point useless. This is one of the reasons why I change the viewpoint from applying t-test to interrupted time-series. $\endgroup$
    – Tito Sanz
    Nov 20, 2019 at 10:20
  • $\begingroup$ Do not know if I understand what you did exactly. The key here is not to use a univariant t-test, but rather consider the information in all 70 categories at once, i.e. exploit the multivariate nature of the problem. If you were finally to perform a test of equality of means, a Hotellings' $T^2$ test rather than a bunch a univariate $t$-tests would be preferable (assuming in both cases the required distributional assumptions hold). $\endgroup$
    – F. Tusell
    Nov 20, 2019 at 11:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.