8
$\begingroup$

Here is a recent Google correlate query:
http://www.google.com/trends/correlate/search?e=internet+usage&t=weekly#

As you can see in the search box at that link, I entered "internet usage" and Google did the rest. It shows a value of 0.9298 as the "correlation" with the query "data mining". However, when I read page 2 of the Google white paper [PDF], it says:

The objective of Google Correlate is to surface the queries in
the database whose spatial or temporal pattern is most highly correlated
with a target pattern. Google Correlate employs a novel approximate nearest
neighbor (ANN) algorithm over millions of candidate queries in an online
search tree to produce results similar to the batch-based approach employed
by Google Flu Trends but in a fraction of a second. For additional details,
please see the Methods section below....

So, my question is:
Is Google using a normal Pearson or Spearman correlation to find this stuff or are they using something else? If so, can you explain the general technique?

==================

Also, notice in the plot that the search for "internet usage" (and "data mining") drops during the summer months and really dives around Christmas. I would guess that kids and their homework have something to do with this.

$\endgroup$
  • 3
    $\begingroup$ Aren't these just Pearson correlation coefficients as reported on the tutorial? (Section Correlated Queries) $\endgroup$ – chl May 27 '11 at 20:16
  • 2
    $\begingroup$ @chl: You're right. I clicked on the FAQ's, but got sidetracked by the Whitepaper. So, they're using correlation of a time series (not the change in the time series). That's like calculating the correlation of a stock price, not returns. That's a little surprising. $\endgroup$ – bill_080 May 27 '11 at 20:23
  • 3
    $\begingroup$ @chl: although your answer is a little obvious, it IS still an answer, and you should make it so, so it can be accepted. $\endgroup$ – naught101 Mar 29 '12 at 9:39
  • 1
    $\begingroup$ @bill_080 Any response from google about why the data differ between the two systems? I actually found this question seeking an explanation related to your observation $\endgroup$ – chandler Nov 21 '12 at 17:22
  • 1
    $\begingroup$ @chandler : Google never answered the e-mail. I played with both Trend and Correlate for a while, but neither were as useful as I expected, so I moved on. $\endgroup$ – bill_080 Nov 21 '12 at 20:21
7
$\begingroup$

As chl points out, the Google Correlate tutorial states that Google Correlate uses Pearson's product-moment correlation coefficient.

They don't mention which language this is implemented in, although Google does use R for some applications, so I'd be guessing that.

$\endgroup$
  • $\begingroup$ Thanks for taking care of filling this thread. (I totally forgot about it, despite your ping :-) $\endgroup$ – chl May 15 '12 at 6:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.