Confusion for 2-way Anova in 2x2 experimental design with planned comparisons

Question

I am really confused regarding 2-way Anovas. I think I did not understand something properly.

Let's say I have a 2x2 experiment design with 2 factorial variables (For the sake of an example let's say Letter and Number so A1, A2, B1, B2 as possible combinations) I would like to do a 2-way Anova since I hypothesise an Interaction between the variables Number and Letter. However, the sample sizes per group are quite small, unbalanced (n = 4-6) and not paired. As part of my experiment design I have 4 planned comparisons:
A1 - A2,
B1 - B2,
A1 - B1,
A2 - B2

Now two questions:

1. As far as I understand I have two orthogonal contrasts?
2. These are the simple main effects or are they interactions?

After the Anova I can NOT do a post-hoc test (because I have planned comparisons and not testing opertunistic?). So I would only test my planned comparison with a t-test with a correction for multiple comparisons (since I am doing more than 3 comparisons).
3. Which t-test should I do (pairwise, two-sample, or not a t-test at all)?
4. Am I doing the planned comparisons regardless of the significance of the Anova?
4.5 . But If so why am I doing the Anova in the first place?
5. Is it okay to just do a t-test with correction, without the Anova?

I looked at the following cross validation questions, but I am sill confused understand:

Calculating 'k' in the bonferroni procedure when there are both post-hoc and planned constrasts

Is it appropriate/acceptable to do planned comparisons for data that will be submitted to a scientific journal?

Thank you for your help in advance!

Answer 1

Welcome to CV! Why do you think you shouldn't do post-hoc comparisons, especially planned ones? I see no reason for why you shouldn't. Post-hoc comparisons are typically conducted with p-value adjustments, which is something you should do. Such adjustments correct for conducting multiple comparisons.

Based on your planned comparisons, you need an interaction effect between factors A and B (otherwise you can't compare A to B). So, you could run a model with A and B and their interaction predicting your dependent variable, and then check your planned comparisons (how exactly you should do this depends on the software and functions you use).

Another thing is that your sample is unbalanced. Typically, ANOVA is not recommended for unbalanced data, but with small differences this is typically not a problem. However, you also have very small cell sizes, so it might be a problem, but it's hard to say whether it is one without knowing more about your variables and hypotheses.