1
$\begingroup$

I want to aggregate Google Trends data relating to separate queries that share a thematic category - e.g. "beer" and "wine". I don't care about the relative popularity of the terms - it is more important to capture the geographical distributions and trends over time - so I can work with the raw 0-100 normalised data from Google.

The problem with aggregating two pre-normalised datasets is, as far as I can see, that sets described by a large peak will be under-weighted in the aggregation. To illustrate consider 2 identical sets except for one having a maximum value twice as high prior to normalisation, e.g:

require(tidyverse)

d = data.frame(x = rep(1:5,2), y = c(1,2,5,3,4, 1,2,10,3,4), set=rep(c('A','B'),each=5)) %>% 
  group_by(set) %>% mutate(y = scales::rescale(y, from=c(0, max(y)), to=c(0, 10)))

ggplot(d, aes(x, y, col=set)) + geom_line(size=1)

This is how Google reports the data. But when we aggregate 2 sets:

aggregate(y ~ set, data=d, mean)
#>   set y
#> 1   A 6
#> 2   B 4

.. set B with originally higher maximum value is diminished in the aggregation.

Are there any statistical approaches to this problem? I thought about re-normalising sets to the same standard deviation, but realised this would inflate the variance in any relatively flat sets.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.