I want to prove that, overall, signal B is correlated to signal A. I was thinking of using cross-correlation (in R) to measure this.
Essentially I have two kinds of signals: signal A is a series of single-valued data describing a particular song; signal B is a series of single-valued data for a user. There are many songs and many users per song, but I do not have the same number of users for every song.
Signal A (song data), for song 1 0.994 0.986 0.955 0.890 0.795 0.650 ... Signal A (song data), for song 2 0.763 0.788 0.787 0.908 0.854 0.901 ... Signal B (user data), for user 1 listening to song 1 75 74.4 73.7 73 72.3 72 ... Signal B (user data), for user 1 listening to song 2 71 72.3 74.9 73 72.5 72.9 Signal B (user data), for user 2 listening to song 2 60.6 60.2 61 60.7 61 59.3 ... Etc.
The series are obviously truncated for this illustration. Again, there are many songs, and not every user listened to every song.
I am interested in whether I can draw conclusions about how well all song data (signal A) can predict all user response (signal B).
Ideally, I would like to capture the cross-correlation in one number (one test statistic for each song), so that I may easily quantify whether there is an overall correlation between the two signals. Using ccf (in R) gives me a value for each lag. For example:
> print(ccf(x,y)) Autocorrelations of series ‘X’, by lag -6 -5 -4 -3 -2 -1 0 1 2 3 4 -0.242 -0.090 0.057 0.197 0.466 0.699 0.896 0.436 0.221 -0.018 -0.116
(Are these values the cross-correlation coefficients?) Also, my data are not stationary. Is there any way (another function?) to test whether signals A and B are correlated across users and songs? One approach would be to average signal B (take the mean user response) for each song, but because there are a different number of users for each song, working with means might be problematic.
So, my main questions again are:
If I perform a cross-correlation for one user data/song data pair, how do I test for significance? Will R give me a correlation coefficient at each lag, or does it only tell me which lag is significant (but not provide any test statistic)? If the latter is the case, will I need to adjust one series of data (to account for the lag) before running a normal Pearson's correlation?
What test may I use when the data are not stationary?
There are a different number of users for each song. For this reason, I can't simply take the average of all users' data for each song (to correlate the mean user data with the song data) - is that correct? Is there a way to test the correlation between signals A and B for each song (across existing users), or must I try to calculate the correlation for each user/song pair individually?
I hope my intent is clear. Thanks for any insight.