0
$\begingroup$

I want to know if someone has some experience working the 'issue' im encountering.

I have a series of arrays, 18, time series on a 14 year period, I want to build a correlation matrix with these data. But I want to check for outliers in each columns, this is done pretty much easily. I do have some outliers on some of the columns. So how i need to tackle the next step? say for example, i need to remove 20 rows in column 2 which accounts for outliers, 50 rows, in column 12. So by definition the whole set of 18 time series, are not of same size anymore, what to do here? Do i need to normalize the rest of the 16 arrays and make them correspond to the common end date of each array in order to do the spearman correlation matrix? I find this solution a bit radical, do you know another one? I use Matlab to do this...

Thank you very much.

ST

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

First, are you sure you want correlations among time series? These are fraught with problems, chiefly that there are many false correlations. E.g. the population of China is highly correlated with the Dow Jones Industrial Average.

Second, if you do want correlations, then instead of deleting outliers you can run a correlation measure that copes better with them, such as rank correlation, provided that the outliers are not spurious data.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks Peter. In fact, I am doing correlations among log stock returns among sectors. Which can give an initial behaviour. Indeed you are right, on the rank correlation, thats why i am using Spearman Correlation instead of the classical Pearson. $\endgroup$
    – Sitingbull
    Commented Oct 8, 2014 at 11:06
  • $\begingroup$ I second Peter Flom's first point. On the second one, you are much better off detecting and deleting outliers than using Spearman.. Finally, it's immaterial whether the outliers are spurious data or not: this is a judgment call. The main problem with them is that a handful of them will drive the results and one typically doesn't want this. Finally, in order to decide on what you do with outliers, you need to find them first anyways. $\endgroup$
    – user603
    Commented Oct 8, 2014 at 11:48
  • $\begingroup$ I think following the rather detailed advice in the thread that User603 links to is the best advice. A robust measure might be best; deleting outliers might be best. It is hard to say without more information. It may be hard to say, even with more information. But I don't agree about "spurious data" - often the investigators can tell that some data are just wrong - they are data entry errors, off by a factor of 10 or something. But other outliers can, often, be very interesting. Plotting the data is essential. $\endgroup$
    – Peter Flom
    Commented Oct 8, 2014 at 12:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.