1
$\begingroup$

I have a question how to analyze mean differences where I am unsure whether I am lacking the statistical vocabulary or simply ignoring something important.

I have a 5 point satisfaction variable (very satisfied to totally unsatisfied) as dependent variable and three group independent variable to predict the satisfaction. The independent variable has three groups: "Top 2 users", "Neutral", "Bottom 2" - these groups are defined according to some other detailed satisfaction questions which are aggregated before (e. g. "Do you like your telephones color? Do you like the camera? etc.). I have used a Oneway test to determine whether or not there are differences in the first place - and the differences are highly significant and very obvious (mean in the Top 2 group at 1.9 and in the Bottom 2 group 4, ignoring the "neutral" group which is not of real interest).

But know my supervisor asked me which of the two groups is more important - e.g. is it more important to gain Top 2 users or to avoid having Bottom 2 users. I am unsure if I can answer the question with the data I have at hand. Do you have an idea if this question is even answerable? I have calculated the effect size, which is quite strong with -0.76, but is this really a good answer to the question at hand?

My method to get the effect size:

df is a dataframe, var is the independent variable, testvar the dependent variable.

getEffectSize <-function(df, var, testvar)
{
  out <- by(df[,testvar],df[,var], stat.desc)

  effectSize <- mes(out[[1]][["mean"]], out[[3]][["mean"]], out[[1]][["std.dev"]], out[[3]][["std.dev"]], out[[1]][["nbr.val"]], out[[3]][["nbr.val"]])

 print(effectSize)
}
Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

From your description I gather that top 2 and bottom 2 groups are independent. If this is so, you want to do a simple independent samples t-test between those groups. This will tell you if the observed difference between the means is 'statistically significant'. But you pretty much already know that.

As for the question: "Which of the two groups is more important?" This is not a question that can be answered with the data you have. What does important mean? The only way to interpret this question in terms of the data you have is to make it into a question of which group, Top 2 or Bottom 2, is further away from Neutral. This can be answered by running a t-test for each of the pairwise comparisons in your data.

I think you need to clarify what is meant by "important."

Improve this answer
$\endgroup$
4
  • $\begingroup$ (I hope my imperfect english does not complicate this discussion) I have also done the t-test (as a posthoc test) - which confirms that there is a difference). My last idea was to compute the absolute difference from the grand mean of the dependent variable and then subtract the Bottom 2 value from the Top 2 value - which gives me a value between 0 (no difference) and +/- 5 - if it's negative the bottom 2 values are more important and vice versa. This gives rather strong results - most values are around -2 to 2.5. Do you see any fundamental problem with that approach? $\endgroup$
    – Christian Sauer
    May 28, 2014 at 14:35
  • $\begingroup$ I guess that is a valid approach to something, I just am not sure what that 'something' is. What question is that supposed to answer? To me it appears that you are asking something like this: "Are Top 2 users further away from the overall mean than Bottom 2 users?" >This gives rather strong results - most values are around -2 to 2.5. Are these values distances of Top 2 and Bottom 2 scores from the grand mean? If so, how does that translate to 'importance'? By the way, your English is perfectly fine. $\endgroup$
    – Lost in transcription
    May 28, 2014 at 15:23
  • $\begingroup$ I think we agree on the question I want to answer. i think thats a good sign. yes, the values are distances from the grand mean. It translates into importance because it shows that dissatisfied (Bottom 2) persons are much more dissatisfied in general (the depenent question) than the Top 2 users. Furthermore, dissatisfaction is much more damaging than satisfaction. A learning could be "Dissatisfied users won't be satisfied with your product, whereas satisfied users MIGHT be satisfied with you, but it's not as clear cut" $\endgroup$
    – Christian Sauer
    May 30, 2014 at 12:28
  • $\begingroup$ I see, I think I have a better hold of the problem now. You have already found that there are meaningful differences between Top 2 and Neutral users, and between Bottom 2 and Neutral users. Now you want to know whether Bottom 2 users are more or less unsatisfied than Top 2 users are satisfied. One way to do this is to run the two t-tests (Top 2 vs Neutral; Bottom 2 vs Neutral), and save the resulting difference scores, and their associated confidence intervals. These estimates will tell you how far each extreme is from the Neutral. $\endgroup$
    – Lost in transcription
    Jun 1, 2014 at 3:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.