What equations are used for calculating a two-way ANOVA across 4 groups consisting of genetic knockouts ($KO$) and wild-type ($WT$) specimens some of which have undergone treatment the rest have been maintained as a control:

$KO_{control}$ X $WT_{control}$ X $KO_{treatment}$ X $WT_{treatment}$

The control groups ($KO_{control}$ and $WT_{control}$ ) each have a sample size of $n=2$.

The treatment groups ($KO_{treatment}$ and $WT_{treatment}$) each have a sample size of $n = 3$

The example that I am working off of only has the formulas for computing the two-way test with groups that have equal sample sizes but I have been made aware that we will need to use a different method for groups with differing sample sizes. I've attempted to look this up but the search space for ANOVA is muddy yielding several conflicting examples. So I've decided to ask you people.

Here are the equations that I've reverse engineered from a colleague's MS Excel spread sheet;

$SS_{total} = (\sum{\sum{X}})^2-\frac{\sum{(\sum{X})^2}}{N_{total}}$

$\bar{X}=\sum{(\frac{N_{X}}{N_{total}}}\cdot \mu{X})$

$SS_{within}=\sum{N_{X}}\cdot (\mu X-\bar{X})^2$


$df = \sum{(N_{X}-1)}$


$tvalue = \frac{ \| \mu KO - \mu WT \|}{\sqrt{\|MS_{within}\|}\cdot(\frac{1}{N_{KO}}+\frac{1}{N_{WT}})}$

$pvalue =$ t-distribution-2-tailed-look-up $(tvalue, df)$


What are the formula for computing a two-way ANOVA across 4 groups with unequal sample sizes?

Specifically for: $SS_{within}$, $MS_{within}$, $tvalue$ and $pvalue$.

A complete answer will have the correct version equations above with sources cited.

If the equations above happen to be correct in then please provide a sources that back that up.

Thanks in advance!

  • 3
    $\begingroup$ Why would asking for formulas be off-topic??? This is clearly on-topic, voting to reopen. $\endgroup$
    – amoeba
    Jul 6, 2017 at 11:44
  • 1
    $\begingroup$ @amoeba, the original post was about programming, and this post has since changed significantly to be in line with the forum's posting guidelines. James Draper, a single sentence changes the entire context and intention of your post. It was a question, in my opinion, about how to program, in python, to achieve your desired result (calculating p-values). Pursuant to the forum guidelines discussed here, I voted to close: stats.meta.stackexchange.com/questions/4846/…. $\endgroup$ Jul 6, 2017 at 15:49
  • 6
    $\begingroup$ And I still submit you are much better off using the python regression or ANOVA libraries rather than "rolling your own" formulas for this. If you are curious as to what the calculations are, any elementary statistics textbook should have them (or at least ones on linear models). See for example, Applied Linear Statistical Models (any edition) by Kutner, et al. $\endgroup$ Jul 6, 2017 at 15:53
  • 2
    $\begingroup$ If you are planning on doing this "for several thousand different measurements," as a comment on one of the answers states, make sure that you use appropriate corrections for multiple testing. $\endgroup$
    – EdM
    Jul 6, 2017 at 17:08
  • 1
    $\begingroup$ Could you please say a bit more about the nature of the measurements that you are analyzing. For example, are these gene-expression measurements of some type? (A comment on a now-deleted answer indicated that you have several thousand of such measurements to analyze, as typically seen in gene-expression studies.) The nature of the data could make a difference to the answer you seek. $\endgroup$
    – EdM
    Jul 6, 2017 at 21:34

1 Answer 1


If you were simply doing a single fixed-effects 2 x 2 factorial ANOVA with unequal numbers of cases among the 4 cells, the formula for partitioning the sums of squares would be fairly simple.*

But you are not doing a single fixed-effects 2 x 2 factorial ANOVA with unequal numbers of cases among the 4 cells. You are doing thousands of such comparisons based on data that themselves hide a good deal of technical and statistical pre-processing. (Your data are presently "relative abundance": relative to what? To a single one of your no-treatment/WT samples? If so, that puts a lot of importance on one single sample. How were differences in mass-tag purity handled? How about differences in peptide-tagging efficiency among samples?)

Proceeding as you propose would be substantially suboptimal.

The tandem-mass-tagged (TMT) proteomics data represent thousands of results (on individual peptides) analyzed in parallel on only 10 samples. Furthermore, you presumably don't care so much about about the peptides detected by the Q Exactive Plus mass spectrometer as you care about the full-length proteins from which the peptides were derived. You may care even more about the biologic pathways in which these proteins participate, to see for example how KO affects the response to treatment at a systems-biology scale.

These are the types of problems that were faced by those performing microarray analysis of RNA expression, now dating back 20 or more years. A large body of work over that time has developed ways to analyze such data efficiently and with a view toward their biological basis, for example in the Bioconductor project. This work now includes workflows for proteomics, which can process the data into forms that nicely fit into the biologically oriented analyses originally developed for microarrays.

Consider the simple issue of estimating within-cell error. Rather than doing this one cell at a time so that the estimates vary wildly from cell to cell, ANOVA pools the within-cell errors among all 4 cells of your design to get a more efficient estimate. The limma package in Bioconductor goes a step further, taking advantage of the data structure (thousands of types of observations, a few samples) to compute moderated standard error estimates based on the entire data set, then using these moderated errors to provide rankings of differential expression for comparisons of interest involving WT/KO and treatment. It includes appropriate statistical correction for the thousands of individual peptides or proteins you would examine. Bioconductor and other bioinformatic systems are also designed to help put together the data on individual proteins into a biological context based on Gene Ontology or biochemical pathway analysis.

So yes, you could write your own code to do a 2-way factorial ANOVA with unequal numbers of cases among the cells. But do you also want to write the code to map peptides to proteins, get moderated standard error estimates within cells, do corrections for multiple comparisons, do the rank-ordering of expression, and map the proteins to pathways? If not, you will be better off learning (or working with someone who already knows) a well vetted and established system for analysis of such data. And you might be better off going back to a level of data processing earlier than the relative expression data, which depend heavily on the question of "relative to what?"

*As the unequal numbers of cases are not related to characteristics of the underlying populations, you could do an unweighted-means ANOVA analysis. Although in general this analysis is only approximate, in the case of factorial designs with 2 levels for each factor (as in your case), Speed and Monlezun (American Statistician 33: 15-18, 1979) showed that unweighted-means provides exact F-tests.

You calculate the within-cell mean square from the deviations of individual observations from their corresponding cell means (in your case with 10-4=6 degrees of freedom). For sums of squares for KO, treatment, and their interaction, you start with the cell means in ANOVA as if they were individual observations, then multiply each sum of squares by the harmonic mean of the numbers of cases per cell (12/5 in your case). From there, standard hypothesis tests and associated p-values apply. I used my 1971 copy of Winer's "Statistical Principles in Experimental Design" as a reference, but any reference on unweighted-means should do.

I'm not sure that this gets around issues of Type I/II/III sums of squares in terms of interpreting the partitioned sums of squares if you were strictly doing a classic ANOVA. You presumably, however, are more interested in simple main effects instead (effect of treatment within each of KO and WT, effect of KO in absence of treatment), which is effectively in this case a set of t-tests that use the pooled within-cell error estimate along with the actual number of observations in the cells being compared.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.