I am currently trying to address violations to ANOVA assumptions. I have used Shapiro-Wilk to test normality, and have dabbled with both Levene's test and Bartlett's test of variance equality. I have since log transformed my data to try and remedy the unequal variances. I reran the Bartlett's test on the log transformed data, and still received a significant p-value, and out of curiosity also ran the Levene's test and got a non-significant p-value. Which test should I rely on?
Probably neither. It would be better to look at your data and see how bad the violations are. Linear models (e.g., ANOVA) are fairly robust to minor violations when the group $n$s are equal. A rule of thumb for heteroscedasticity is that the maximum group variance can be as much as 4 times the minimum group variance without too much damage to your analysis. If you are worried that there may be violations, an even better approach is to simply use analyses that are robust to the possible violations from the start, rather than trying to detect violations and then make decisions based on that1.
For what it's worth, Wikipedia says that Bartlett's test is more sensitive to violations of normality than Levene's test. So you may have non-normal data instead of heteroscedastic data. Again, a more robust analysis may be preferable2.
1. See: A principled method for choosing between t test or non-parametric e.g. Wilcoxon in small samples.
2. For various ways of dealing with problematic heteroscedasticity, see: Alternatives to one-way ANOVA for heteroskedastic data.
2$\begingroup$ ...fairly robust to minor violations with equal Ns. $\endgroup$– JohnJan 27, 2015 at 21:59
$\begingroup$ And then there's the issue that you might have strong reason to believe that the samples come from populations with roughly equal variances... Which is what the tests of robustness are based on. $\endgroup$– JohnJan 28, 2015 at 2:47
$\begingroup$ Can i visually check the range of variances using diagnostic plots? $\endgroup$– ClariceJan 28, 2015 at 3:53
$\begingroup$ Sure, @Clarice. Any number of plots will help with that. You can make a scatterplot with the dots arrayed vertically within category levels marked on the x-axis, then you can see how they compare. You could also try boxplots, eg. $\endgroup$ Jan 28, 2015 at 3:59
For a less sensitive test for non-normal conditions than Levene's test at least sometimes use Conover's test, AKA squared ranks non-parametric test. I have found this to be at least sometimes preferred to Bartlett's test in the Mathematica implementation of the VarianceEquivalenceTest.
Here is a list of variance tests methods and assumptions copied from the Variance Equivalence link above
Bartlett normality modified likelihood ratio test BrownForsythe robust robust Levene test Conover symmetry Conover's squared ranks test FisherRatio normality based on variance ratio Levene robust,symmetry compares individual and group variances
What should be obvious from that list is that the violations of assumptions are testable, although the Mathematica documentation is not specific as to how, for example, the Conover symmetry test is being performed, or even why one tests for symmetry. And, so far no one has answered that question.
So, the answer to the OP question is that only testing of conditions can suggest which method is preferable in any particular case. Moreover, if all 5 tests are attempted, and are not excluded because of violation of assumptions, then one can generally distinguish between better and worse answers with whichever answers are generated.
As a worst case, one can perform Monte Carlo simulation using known truth values to explore which conditions lead to what probabilities. But, without more information as to the problem itself, the question cannot be answered in terms of the OP's data set. If the OP wants a data oriented specific answer, please provide the data.
2$\begingroup$ Conover's test is a reasonable suggestion here. But you shouldn't mix an answer to this question w/ a new question of your own & a request for feedback (from whom?) regarding parts of your answer, or asking for your suggested edit to be approved. $\endgroup$ Feb 9, 2017 at 21:49
$\begingroup$ @gung Yeah fine, changed it to be more immediately useful. $\endgroup$– CarlFeb 9, 2017 at 23:31