Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have the following data set that I need to display on the heat map:

[ 30, 15, 66, 7, 9999, 78, 42, 132 ]

So if I map the values to the color scale using a linear function I only see the spike while the rest of the values I cannot tell apart because the difference between them is insignificant in the scale of that spike.

I tried a log function but it didn't get a good looking picture either.

I don't care about accurate correspondence between the value and the color intensity.

Is there a way to map the values to the color scale so that the difference between all values was visible? Can I mitigate that spike somehow? I am asking about established approaches or algorithms addressing problems like this.

share|improve this question

migrated from programmers.stackexchange.com Mar 27 '13 at 19:42

This question came from our site for professional programmers interested in conceptual questions about software development.

    
Consider mapping the data that is within N standard deviations of the mean to the heat map with special colors/symbols for the outliers. –  MichaelT Mar 27 '13 at 14:45
    
You could try to renumber them by their position if you would sort them ([3,2,4,1,8,6,5,7]). If you get more distinct numbers than colors, you can combine multiple as one (either those whose value is close or log(value) is close (or some other function). –  user470365 Mar 27 '13 at 15:10
    
Very closely related: stats.stackexchange.com/questions/51792/…. Does this answer your question? –  whuber Mar 27 '13 at 19:44
    
I would have set a treshshold, computing another vector v such that vi= min(200, xi). –  lmorin Mar 27 '13 at 19:53

1 Answer 1

There are many different ways of handling outliers, as well as answers to similar questions here already. I would recommend taking a look at

The first deals with this problem in R. Taking the approach of graphing the non-outliers with a heatmap, then overlaying the outliers and annotating the legend to describe that this is what was done.

The second contains links to general solutions and approaches to outliers.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.