Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In R, I am trying to write a function to subset and exclude observations in a data frame based on three variables. My data looks something like this:

data.frame':   43 obs. of  8 variables:
 $ V1: chr  "ENSG00000008438" "ENSG00000048462" "ENSG00000006075" "ENSG00000049130" ...
     $ V2: chr  "ENST00000008938" "ENST00000053243" "ENST00000225245" "ENST00000228280" ...
 $ V3: chr  "ENSP00000008938" "ENSP00000053243" "ENSP00000225245" "ENSP00000228280" ...
     $ V4: chr  "PGLYRP1" "TNFRSF17" "CCL3" "KITLG" ...
 $ V5: chr  "19" "16" "17" "12" ...
     $ V6: chr  "q13.32" "p13.13" "q12" "q21.32" ...
 $ V7: int  46522415 12058964 34415602 88886566 8276226 150285143 29138654 76424442 136871919 6664568 ...
     $ V8: int  46526304 12061925 34417515 88974238 8291203 150294844 29190208 76448092 136875735 6670599 ...
>

What I am trying to do is this:

input = function(x) {
        pruned1 <- subset(x[-which(x$V5 == 1 & x$V7 > 113818477 & x$V8 < 114658477), ])
        pruned2 <- subset(pruned1[-which(pruned1$V5 == 1 & pruned1$V7 > 192461456 & pruned1$V8 < 192549912), ])
        # and so on
        }

So I want to take my original data frame, exlude observations that fulfil all three of the subsetting conditions, make a new object of the remaining observations (pruned1) and exclude any observations that fulfil a new set of condidtions and so on.

The problem I am having seems to be the third condition in the first subsetting, as this returns:

'data.frame':   1 obs. of  8 variables:
 $ V1: chr NA
     $ V2: chr NA
 $ V3: chr NA
     $ V4: chr NA
 $ V5: chr NA
     $ V6: chr NA
 $ V7: int NA
     $ V8: int NA

While only using the first two conditions returns:

'data.frame':   38 obs. of  8 variables:
 $ V1: chr  "ENSG00000008438" "ENSG00000048462" "ENSG00000006075" "ENSG00000049130" ...
     $ V2: chr  "ENST00000008938" "ENST00000053243" "ENST00000225245" "ENST00000228280" ...
 $ V3: chr  "ENSP00000008938" "ENSP00000053243" "ENSP00000225245" "ENSP00000228280" ...
     $ V4: chr  "PGLYRP1" "TNFRSF17" "CCL3" "KITLG" ...
 $ V5: chr  "19" "16" "17" "12" ...
     $ V6: chr  "q13.32" "p13.13" "q12" "q21.32" ...
 $ V7: int  46522415 12058964 34415602 88886566 8276226 150285143 29138654 76424442 136871919 6664568 ...
     $ V8: int  46526304 12061925 34417515 88974238 8291203 150294844 29190208 76448092 136875735 6670599 ...
>

Doing this manually is not an options, since I have a large number of input files that I would like so prune in the same manner.

Thanks in advance for any help!

share|improve this question
add comment

3 Answers 3

up vote 2 down vote accepted

You are using subset wrongly. The syntax of subset is

    pruned1 <- subset(x, !( V5 == 1 & V7 > 113818477 & V8 < 114658477) )

("!" is the negation operator)

and not

    pruned1 <- subset(x[-which(x$V5 == 1 & x$V7 > 113818477 & x$V8 < 114658477)

Hope that helps

PS You might want to look into whether using '&' or '&&' fits your situation best, I can't remember if that will cause a problem.

share|improve this answer
1  
In this case, I don't think that using && makes sense, see ?"&" and R: Logical operators: Beware of the difference between “&&” and “&” . –  Bernd Weiss Mar 14 '11 at 15:54
    
I think you meant pruned1 <- subset(x, !(V5 == 1 & V7 > 113818477 & V8 < 114658477)) right? pruned1 should exclude observations for which V5==1 & ... –  lockedoff Mar 14 '11 at 16:07
    
Yes I wanted to exclude the observations that fulfilled the conditions. So this was exactly what I meant, thank you. The function works beautifully now. –  Caroline Brorsson Mar 17 '11 at 8:41
add comment

If you are using which, then subset is unnecessary. If we have a logical condition these are equivalent in R:

data[cond,]
data[which(cond),]
subset(data,cond)

The benefit of the subset is that you do not need to use $ to get to the variables you are subsetting on.

Furthermore if you do successive subsetings it makes more sense to concatenate all the conditions and then do subseting. So if you have the condition 1 and condition 2, which themselves may be complicated conditions then instead of

pruned1 <- data[cond1, ]
pruned2 <- pruned1[cond2, ]

Do

cond <- cond1 & cond2
pruned2 <- data[cond,]

The main benefit of the second approach that it is faster and cleaner, since you will not be creating new objects.

share|improve this answer
    
Thank you for clearing my confusion about which and subset. I was doing it more complicated than it was. The tip about my successive subsetting is also speeding up my analyses, which is great. –  Caroline Brorsson Mar 17 '11 at 8:31
    
@Caroline, you are welcome. –  mpiktas Mar 17 '11 at 10:14
add comment

Another way to do this is

y <- 
    x[
        !(x$V5 == 1 & x$V7 > 113818477 & x$V8 < 114658477) &
        !(x$V5 == 1 & x$V7 > 192461456 & x$V8 < 192549912),
    ]

Also, V5 is a character variable, so x$V5 == “1” might be preferable -- e.g., if you have any codes that are actually formatted as " 1", you will need to take care to specify x$V5 == “ 1”. @ulvund & is necessary for this application because it returns element-wise logical AND, whereas && “evaluates left to right examining only the first element of each vector. Evaluation proceeds only until the result is determined” (?Logic).

share|improve this answer
    
I haven't even noticed that V5 was a character variable, thank you for pointing it out. Somehow it still worked, but it's good to know in downstream analyses. I am still confused about &&, but I am saving that for later. –  Caroline Brorsson Mar 17 '11 at 8:56
    
@Caroline You can check everything easily in R: "1"==1 returns true (T) but " 1"==1 returns false (F). c(T,F,T) & c(F,F,T) returns F,F,T as expected but c(T,F,T) && c(F,F,T) returns F because it only compares the first element in each vector. –  lockedoff Mar 17 '11 at 14:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.