How safe is that to ignore homogeneity of variances test and continue with Post hoc test?

Question

I want to know if average time people spending on their favourite social media are statistically different. I consider here five groups, and the total number of participant is 700 persons.

I have already performed the Shapiro-Wilk normality test, on each group (Facebook, Twitter ...) to see if the time people spend on social media follows a normal distribution, according to the test, the null hypothesis fails so the time isn't normally distributed. Though, I decided to consider the time as a normal distributed parameter, because the average time is approximately normally distributed as $n=700$ is large enough.

Next, I carry out the 'Test of homogeneity of Variances', according to the the results the equality of variances is ruled out.

Meanwhile, I looked at the 'Robust test of equality of means' or 'Welch and Brown-Forsythe', both tests results in p-value $0,000$.

Now, I want to know if I can trust the results of Post hoc? Or I frequently violated the rules? If so how I can eventually compare means?

I have a very few experience in data analyses, so I apologise here for carelessness of my argument.

Answer 1

As you can see from the question in my Comment, I'm just guessing that you have 700 people altogether, with approximately $n_i = 140$ in each (social media) group.

If that is true, Shapiro-Wilk normality tests on individual groups of over 100 each should pick up the degree of non-normality that would invalidate a Welch (separate-variances) ANOVA.

You say you have found significant differences among the five groups with overall a tiny overall P-value.

In that case it should be fruitful to look at ad hoc paired comparisons using two-sample Welch t tests. You could mitigate 'false discovery' by using Bonferroni criteria.

For example, if you do all ${5 \choose 2} = 10$ comparisons, you could look for comparisons significant at the $0.5\%$ level. (However, you may find that you don't need to do all possible comparisons in order to get a good idea what the pattern of significant differences might be.)

Note: If you are using R, perhaps see this link for the functions to use, and some ideas on methods (including, but not limited to, 'Bonferroni') avoiding false discovery in making ad hoc comparisons.

Answer 2

Look at

           mean st.dev        cv
Snapchat   93.6   94.6 1.0106838
Facebook   54.7   62.3 1.1389397
Instagram  78.1   78.0 0.9987196
Twitter   101.6   94.0 0.9251969
Andere     76.3   87.0 1.1402359

The coefficient of variation is close to a constant, so that might be a better assumption than constant variance. So a glm (generalized linear model) with constant coefficient of variation could be a possibility, that is, a gamma glm. For this data, maybe an identity link function.

But your data is quite strange. Average time on Twitter almost double that on Facebook? Wouldn't have guessed that!

Answer 3

There are several possibilities.

Since you measure total times for each respondent the values are at least zero and are likely heavily skewed to the right. This suggests that a transformation to reduce skewness followed by normal theory tests would be useful. You could consider taking the log of each time. If you have a lot of values at zero, you could consider taking the square root of each time. (These do not differ greatly.) The transformed values should be more normally distributed than the observed times, but there is no assurance that they will be as well-behaved as you desire. Even so, if you reduce the ratio of the standard deviation to the mean (in each group) by a substantial amount then normal theory tests will work better.

Another choice is to use a nonparametric procedure based on ranks. If you do an omnibus test, corresponding to ANOVA, you will want the Kruskal-Wallis test, possibly with multiple comparisons of the individual groups. If you compare two groups at a time , you will want to Wilcoxon or Mann-Whitney test - they are the same. You could adjust the p-values for multiple comparisons in either case.