How to know which statistical test to use?

Question

I'm working on a project and was wondering how I could perform statistical tests between two correlations to prove significance.

The background of my project is that I was trying to see if Twitter sentiments influence a certain market more than the other. After collecting data and calculating the correlation values using three methods (Pearson, Kendall, and Spearman) I found that the difference is very small. However, I feel that simply saying "the difference is small" is not convincing and was wondering if there were specific statistical tests for this purpose.

I've done searching and have seen that people use p-value tests quite often, but I'm not sure if that would be appropriate for my case, as I'm also not familiar with statistical testing methods. I was hoping if anyone more knowledgeable would be able to give me some tips or pointers.

The data that I'm specifically using looks like this:

To briefly explain the data, TextBlob and NLTK Vader are two sentiment analysis tools I used. Within these two methods, I applied time lag to the data (one day forward and one day backward) and you can also see that I varied calculating correlation. X and Y are two separate markets and the correlation is between price and sentiment. The difference is the difference between these two.

My original hypothesis was that market Y would have higher correlation, but apparently that is not true judging by the difference.

What might be some ways that I could formally prove the significance of my finding?

Thank you.

Answer 1

You could calculate the 95% confidence intervals of the correlation coefficient values for markets X and Y.

The correlation coefficient value you have calculated is a best estimate based on your sample of data. The 95% confidence interval tells you, based on your data, what range of correlation coefficient values you could expect to get 95% of the time, if you were to repeat the experiment many times. So hopefully the 'true' population value will lie somewhere within that range.

Once you have calculated the confidence intervals for your correlation coefficient for both markets X and Y, check if they overlap. If they do, then they are not significantly different. If they do not overlap, then they are significantly different.

You can calculate confidence intervals for Pearson's correlation coefficient (usually quite straightforward), or for Kendall's Tau or Spearman Rank correlation coefficients, but it might involve bootstrapping.

Please note that there is no need to use all three correlation tests - choose just one, so long as you are sure that your data meets its assumptions. Generally, choose the most powerful test (that is, the one with the greatest power to detect a true difference) for which your data meets the test assumptions.