Comparing two sets of data for similarity

I have two independent sets of gene expression data (biological replicates) that were generated at the same time. Each gene is sampled dozens of times on the machine (technical replicates). Data are filtered for transcripts having expression values above background (p-values less that 0.05). I then export everything for analysis in Excel. However, the raw expression values can vary between the two samples. The majority seem to be rather close but many are far enough apart to make me question their reliability.

What would be the best way to compare the two values ($X_1$ and $X_2$) such that I can create a new filtering criteria to remove these potentially less reliably sampled transcripts?

I only have two datasets, and I'm uncertain as to whether I can use standard deviations in any filtering. My general thinking is that if there is less than a 20% difference between the two values I can keep those that pass. But I don't know what that would mean. Take these genes as an example:

$$\begin{array}{ccc} & X_1 & X_2 \\ \mbox{Strap} & 5554.15 & 5262.48 \\ \mbox{Cops8} & 1762.63 & 2317.22 \\ \end{array}$$

As you can see, the Strap values are pretty close, the Cops8 are quite a bit different. Does a "% difference" criteria make sense? Something like this:

$$\mbox{%diff} = \frac{\left|X_1 - X_2\right|}{(X_1+X_2)/2}$$

If I calculate a coefficient of variation I get a much different value. I don't want to exclude more genes than I have to, if I can avoid it.

• If you created a histogram of this % difference metric, what does it look like? Is it roughly normally distributed or are there outliers or fat tails? – David Robinson Jan 16 '13 at 4:23
• If I calculate the normal cumulative distribution for ALL the genes the right-hand tail is obscene. This is because there are lots of genes with extremely low values, some even negative. If I remove the genes that have negative values in either replicate then it looks more normal but still has a wide tail. This is what I want to improve upon. While in general the genes with low expression values have larger %diff and higher p-values (>0.05), that isn't the case for all of them. Thus, I want to have the soundest exclusionary criteria. dropbox.com/s/c9loeh91us8lbu7/data.png – captainentropy Jan 17 '13 at 3:06
• Try histogramming the log of the absolute value of the % diff- how does that look? – David Robinson Jan 17 '13 at 3:07
• There might already be a package to analyze this. Take a look at this site; bioconductor.org/packages/release/BiocViews.html#___Software – rnso Jun 3 '15 at 17:25

How many genes are sampled in either data set you've obtained? When the number of genes is large, it shouldn't be surprising that a gene may be highly differentially expressed in one replicate but not in another. This is a consequence of multiple testing. When you filter genes according to a $p=0.05$ statistical significance level, you have a 5% chance of making a type I error for any given gene. When averaged over several dozens or hundreds of genes, the chance of including at least one erroneous gene is greatly multiplied.