I want to aggregate Google Trends data relating to separate queries that share a thematic category - e.g. "beer" and "wine". I don't care about the relative popularity of the terms - it is more important to capture the geographical distributions and trends over time - so I can work with the raw 0-100 normalised data from Google.

The problem with aggregating two pre-normalised datasets is, as far as I can see, that sets described by a large peak will be under-weighted in the aggregation. To illustrate consider 2 identical sets except for one having a maximum value twice as high prior to normalisation, e.g:

require(tidyverse)

d = data.frame(x = rep(1:5,2), y = c(1,2,5,3,4, 1,2,10,3,4), set=rep(c('A','B'),each=5)) %>% 
  group_by(set) %>% mutate(y = scales::rescale(y, from=c(0, max(y)), to=c(0, 10)))

ggplot(d, aes(x, y, col=set)) + geom_line(size=1)

This is how Google reports the data. But when we aggregate 2 sets:

aggregate(y ~ set, data=d, mean)
#>   set y
#> 1   A 6
#> 2   B 4

.. set B with originally higher maximum value is diminished in the aggregation.

Are there any statistical approaches to this problem? I thought about re-normalising sets to the same standard deviation, but realised this would inflate the variance in any relatively flat sets.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.