Two-way ANOVA when data is non-normally distributed

Question

I'm trying to run a two-way ANOVA on a 375-sized dataset. I'm trying to navigate my way through the assumptions but need some assistance. Specifically, I have two groups as the IVs (1: Boys and Girls; 2: Yes and No), and a real number between 0 and 1 as the DV.

Firstly, I know there is no assumption related to equal sample sizes, but mine are really quite different:

Girls.
Yes = 190.
No = 58

Boys.
Yes = 102.
No = 25

Assuming the fact that one group has 190 and another only has 25 is OK (it may not be?), I am having some difficulty with the normality assumption. Both 'boys' subgroups have a low skewness (less than double the std. error) and a non-sig Shapiro-Wilk stat. However, the two 'girls' groups have a negative skewness of 0.8 and 0.3 respectively, and a sig SW.

I have tried a log-transform, however this does not resolve the issue. Is there anything else anyone can think of? With the info provided, should I abandon parametric tests here in favour of non-para, or is OK to continue with these (hopefully) minor shortcomings?

Edit: To provide some more clarity around the problem itself, the DV is a rating score regarding how similar the speech of the person (in this case girl or boy) is to another girl or boy (where theoretically 1 is the absolute repetition of speech, and 0 is the absolute divergence) - all DV values are between 0.75 and 0.95. The 'yes', 'no' IV is merely how they answered a particular question in a previous data collection exercise.

I have initially run a t-test looking at the differences between the mean of girls and the mean of boys, and wanted to further break it down using this additional question variable

Answer 1

With respect to different numbers in each group: although standard textbook presentations of ANOVA use equal numbers in each cell, that's not generally a substantial problem and modern linear-modeling software handles it well.

With respect to normality: there is a school of thought that normality testing is essentially useless at this stage of a study. With a large enough group you will almost always find violations of normality in real-world data. You'll note that the apparent non-normality problem is seen in girls, who outnumber boys by about 2 to 1. With such a restricted range of dependent-variable (DV) values, my initial reaction in a comment that normality shouldn't even be suspected is substantially alleviated.

With respect to data transformation: with all values fractional and not including 0 or 1 exactly, consider a logit transformation. Think of your score as being equivalent to the probability p of a perfect match to the reference individual. Then transform with $\log(p/(1-p))$. That would put your DV values on a scale from about 1 to 3 while stretching out the top of the scale.

With respect to the risks of proceeding in the face of apparent non-normality: the discussion on this page shows that there is no simple rule to follow with respect to violation of normality assumptions. As what you are modeling is something essentially the same as a failure rate, however, you probably should consider the caution in this answer on that page.

With respect to non-parametric tests: if you move to a non-parametric test you can still look for differences between boys and girls and between those who answer Yes versus No, but there will be no good way to look at interactions between gender and answer. As the answer linked in the previous sentence suggests, however, you might be able to analyze your data with an ordinal logistic regression regardless of normality in a way that allows you to evaluate that interaction.