# How do I compare collections of samples for differences?

I got data of the speech length distribution (number of words per speech) of a Speaker in a tv series per episode in my table.

E.g.:

Length Episode1 Episode2 Episode3 | Episode4 Episode5...
1      14       12       68       | 10       8
2      7        61       58       | 22       23
3      14       42       6        | 13       39
4      54       12       14       | 2        12
5      14       18       38       | 41       11
....


I want do compare a group of episodes, lets say in this example 1-3 with the Episodes 4-5 and find out if there is a significant difference in how the speaker speaks after a certain plot break.

What kind of test do I have to do, to find out such a difference!?

FYI: In particular I want to see if the Character Raj uses a different kind of speaking after he overcome his selective mutism after season 6 episode 24 from the big bang theory.

EDIT: I tried with the Kullback Leibler in R as suggested by Hossein. I don't get it to work with the normalized vector:

> a
[1] 271 297 315 332 352 321 309 313 265 212 255 198 192 178 160 150 139 105 107  94  89  81  88  84  59  66  48  53  38  46  28  25  17  34  36  26  27  13  14  16  11  15  13  13   9   6   7   5   2   5   4   4   2   3   7   3   2   1
[59]   3   2   2   2   2   2   1   2   2   2

> b
[1] 46 49 52 51 54 63 59 58 43 33 40 32 42 34 21 27 24 22 17 18 13 10 20 18  8  9  8  9  8  7  6  5  1  7  8  9  6  2  2  3  2  3  1  1  2  1  2  1  1  2  0  0  1  0  1  0  2  1  1  0  0  0  2  0  1  0  0  0

> KL.plugin(normalize(a),normalize(b))
[1] Inf
Warnmeldung:
In KL.plugin(normalize(a), normalize(b)) :
Vanishing value(s) in argument freqs2!

> KL.shrink(a,b)
Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0069
Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0409
[1] 0.01888402
attr(,"lambda.freqs1")
[1] 0.006945795
attr(,"lambda.freqs2")
[1] 0.04085443


I don't think the result of the KL should be Inf for my two vectors. Or is this actually the right answer?

Can anybody explain me what exactly KL.shrink of package entropy does and how to interpret the 3 result values?

• I don't understand the question. You want to compare (10,8) against (13,39)??? – SmallChess Mar 4 '17 at 15:17
• Here each speech is a sentence? – Hossein Mar 4 '17 at 15:45
• No, I want to compare group A (whole data of episode 1-x) with group B (whole data of episode x+1-lastEpisode) – Igle Mar 4 '17 at 16:01
• @Hossein one cell is the number of times a certain speech length appears in an episode. E.g. Episode 1 contains 14 speeches of length 1, 7 of length 2 and so on – Igle Mar 4 '17 at 16:02
• One speech is from the moment a speaker starts speaking until another speaker starts speaking. So the dialogue 'hi! how are you? - I'm good and you? How are you? - I'm fine too' would be 3 speeches. So a speech can consist of several sentences or also not more than a single word like 'hi' – Igle Mar 4 '17 at 21:23

A nice question! Here is my idea to solve it:

Consider you want to compare speeching habits of a person in the episodes 1-3 with her/his habits in the episodes 4-5. For each speech length, compute the average frequency over different episodes in each category. For example, for your table, you get:

Length Episodes1-3 | Episodes4-5
1      31.3    | 9
2      42      | 22.5
....


By normalizing each column (dividing each number by the sum of numbers in that column) you will have a probability distribution for each episode group (one for episodes1-3 and one for episodes4-5). Then, to compare two different distributions, you can use standard methods such as the Kullback–Leibler divergence.

• This is a contingency table approach. – Michael R. Chernick Mar 4 '17 at 21:44
• Okay, the idea seems appropriate but i can not get it to work in r. Could you have a look at my code which i append to my question!? – Igle Mar 9 '17 at 8:28

try with this:

example with two variables: observed counts for two random variables;

y1 = c(4, 2, 3, 1, 10, 4)

y2 = c(2, 3, 7, 1, 4, 3)

Then apply empirical Kullback-Leibler divergence

KL.empirical(y1, y2)

you will get:[1] 0.2232368

From what I understand from the source code: https://rdrr.io/cran/entropy/src/R/KL.plugin.R

you get the "vanishing value" error because one of your values is not >0. If this is the case, you should apply smoothing (https://en.wikipedia.org/wiki/Smoothing)

For example, you could convert all 0s into 0.00000000001. But before you choose a smoothing value, try to understand why smoothing is needed, if it is needed in your case.

Cheers, Marina