Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In R, given a set of numerical data, how do I decide the top 1% of these data, and replace them with a new value, say, e.g. $10^{10}$

share|improve this question
add comment

1 Answer

up vote 10 down vote accepted

You could do this:

### x will be whatever variable you want to do this to. I'm replacing 
### the top 1% with 1e10. 
qnt = quantile(x,.99)
w = which(x >= qnt) 
x[w] = 1e10

More generally implemented as a function:

### x is the input data. This function replaces the top 'perc' percent
### with the value 'rp'. 
replace.top = function(x, perc, rp)
{
   qnt = quantile(x,1-perc)
   w = which(x >= qnt) 
   x[w] = rp
   return(x)
}
share|improve this answer
7  
Or, even more concisely (densely) as a one-liner x[x>=quantile(x,0.99)] <- 1e10. –  cardinal Feb 6 '12 at 3:10
5  
(+1) As a function: replace.top <- function(x,p,r) { x[x>=quantile(x,1-p)] <- r; x} –  cardinal Feb 6 '12 at 3:15
    
(+1, cardinal) Is there anything more computationally efficient about what you've written (or are there just bonus points for making it a single line)? –  Macro Feb 6 '12 at 3:22
1  
Note also that if there are a very large number of ties in x, then basing the replacement on quantiles might not work as expected. An alternative would be x[rank(x,ties.method="first")>=(1-p)*length(x)] <- r but in the "typical" case where the values of x are fairly "continuous", I think the previous approach using quantiles is better. –  cardinal Feb 6 '12 at 3:53
3  
The answers and comments above are good answers. Depending on what the questioner is ultimately looking for @Luna may also be interested in the literature on winsorized means and variances and that there are several implementations in various R packages (eg in Wilcox's WRS package associated with his excellent book Modern Statistics for the Social and Behavioral Sciences.) –  Peter Ellis Feb 6 '12 at 10:14
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.