Statistical significance in time series (python)

I have a list of tweets which are ordered by twitting date, the date range spans over a month. The tweets are categorised in two groups, the first group sentiment is 'sad', the second group is 'other'. I created a time series which shows the number of 'sad' and 'other' tweets per hour. Usually, when the number of tweets increases, the portion of 'sad' tweets increases too. However the increase in 'sad' tweets may not be significant, it can be just the result of the increase in total volume.

I want to see for each hour if the change in the number of 'sad' tweets is statistically significant compared to the overall 'sad' tweets. I think the null hypothesis will be, for a given hour the number of 'sad' tweets is not significantly different than all 'sad' tweets.

I use python (mostly pandas, scipy library) for coding/analysis. I know just very little about statistical significance, so please bear with me if the question is very simple! I will appreciate if you can give me some pointers (if the method exists in a standard python library that would be great).

• "Statistically significant" makes sense only with respect to a null hypothesis. Please state that hypothesis in your question.
– whuber
Mar 28, 2020 at 13:23

Your question is not completely clear and I can't comment because I don't have enough reputation but I'll do some assumptions here please correct if I'm wrong.

So your data (time series) looks like this:

Each timepoint is an hour with a timespan on one month. For each timepoint you have a number of sad tweets and a number of other tweets.

So your data looks a bit like this:

timepoint/date  numberofsadtweets numberofothertweets

2020-01-01,00:00       200                800

2020-01-01,01:00       300                900

etc....


Now you ask: "I want to know for each hour if the number of sad tweets is significant".

The thing about significance is, that it has to relate to a hypotheses + is used when you can't measure the full population. When you want to know if the number of sad tweets is significant my question is what hypothesis are you testing and number of sad tweets significant to what? What are you comparing this with?

I assume you might want to know for each hour if there are significantly more sad tweets than other tweets? This question can easily be answered without significance. You can just look per hour which group has the most tweets. That would be your answer.

However I can assume this was not what you mean. I guess you might want to know if on certain hours there are significantly more sad tweets compared to other hours?

So now your hypotheses is "on certain hours the number of sad tweets is significantly more than other hours".

Now significance and statistics come into play because you sampled one month and not the whole history and future of tweets to come.

What you should do to see if there are more sad tweets on certain hours than others:

Firstly, I don't think one month of tweets is enough for this analysis. Because you will have 24 groups (hours) with only 30 measurements per group(hour). In common practice this is not enough data for this kind of analysis. You might want to make bigger groups by for example taking two hour timepoints. You'll then have 12 groups with 30 measurements. Or take an extra month.

Now lets continue with the testing:

First you might want to correct for total number of tweets. This is done when on certain hours there are way more tweets than other hours so there will be automatically more sad tweets. If you don't mind about this effect its also fine but I think you'll want to correct for this.

One way to do this is to calculate for each hour the percentage of sad tweets of the total number of tweets in that hour.

Now we've corrected for total number of tweets we can compare groups.

Here I highly advise you again to specify you hypotheses or make less groups. With 24 hours you're making 24^2 - 24 : 2 comparisons. With each comparison you're losing power. Just search bonferonni-holm correction and you'll know what I mean.

I'd say make 4,3 or even 2 groups. So you can compare morning afternoon and night for example. This way you'll only have at most (4^2 - 4) : 2 = 6 comparisons.

Okay now the testing:

At the base of this comparison is the t-test. Here you have 1 categorical variable and one numeric variable. The simplest test is for example when you compare morning to evening. your grouping variable is then "part of the day"(morning/evening) and your numerical variable is "percentage of sad tweets". If the test is then significant that indicates that there is a difference in percentage of sad tweets between morning and evening.

If you want more than 2 groups you need a slightly different approach but the idea is the same.

You'll do a one way anova to see if there is any difference between groups. You use hours here as independent (categorical/grouping!) variable and percentage of sad tweets as dependent variable. If the anova is significant, you can do post-hoc t-tests to see which groups are significantly different.

Another way to approach this is to see if you can model the percentage of sad tweets with time series analyses. But this is very different and I think you don't want it.

so in short:

• make less groups
• correct for total number of tweets
• check assumptions for one way anova
• do one way anova
• do post hoc t-tests

I hope my rambling made some sense!

• Thanks a lot, @B.Bram for your answer. I did some edit in my question, I am not sure if it makes sense. All I want to see for a given hour, if the number of 'sad' tweet is increased it is not because of the total number of tweets increased but in fact, at that hour the number of 'sad' tweets are more than usual ( regarding all tweets in the dataset). Hope that makes sense! Mar 28, 2020 at 15:04

oh dear I went totally the wrong way there. The answer to your question is much simpler now you've clarified. It is still not very clear why you would want to do a test like this but here's how you do it:

https://www.universalclass.com/articles/math/statistics/calculate-probabilities-normally-distributed-data.htm

What you might need to do still is correct for the total amount of tweets like in my other answer. This is your choice. Also it makes no sense to me to do 30 days x 24hours = 720 statistical tests because you still need to correct for this with something like bonferroni holm.

Now the test:

What you want to do is determine the mean and standard deviation of your distribution of number of sad tweets. You also need to do some assumptions about the shape of this distribution. For you I hope it is normally distributed otherwise this get a bit messy.

So first calculate your mean and standard deviation. Look up the formula its pretty easy to do.

Then you can calculate the Z score of each data point by subtracting the mean and dividing by the standard deviation.

The Z score comes with a probability that indicates the probability to find the data point in the given distribution. If that probability is less than 0.05 or any other set threshold you might choose you can say that point has a significantly different value compared to the given distribution.

• Thanks for the answer. Is there a workaround for the not standard distributed data, as I think my data does not have a normal distribution. Mar 28, 2020 at 22:49
• Yes there is do you know what shape the distribution of your data has? Also try google what to do if data isn't normally distributed. Mar 29, 2020 at 12:16