2
$\begingroup$

I want to aggregate Google Trends data relating to separate queries that share a thematic category - e.g. "beer" and "wine". I don't care about the relative popularity of the terms - it is more important to capture the geographical distributions and trends over time - so I can work with the raw 0-100 normalised data from Google.

The problem with aggregating two pre-normalised datasets is, as far as I can see, that sets described by a large peak will be under-weighted in the aggregation. To illustrate consider 2 identical sets except for one having a maximum value twice as high prior to normalisation, e.g:

require(tidyverse)

d = data.frame(x = rep(1:5,2), y = c(1,2,5,3,4, 1,2,10,3,4), set=rep(c('A','B'),each=5)) %>% 
  group_by(set) %>% mutate(y = scales::rescale(y, from=c(0, max(y)), to=c(0, 10)))

ggplot(d, aes(x, y, col=set)) + geom_line(size=1)

This is how Google reports the data. But when we aggregate 2 sets:

aggregate(y ~ set, data=d, mean)
#>   set y
#> 1   A 6
#> 2   B 4

.. set B with originally higher maximum value is diminished in the aggregation.

Are there any statistical approaches to this problem? I thought about re-normalising sets to the same standard deviation, but realised this would inflate the variance in any relatively flat sets.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.