1
$\begingroup$

A colleague of mine has asked me for help on this problem, but I am confused with it as well, the main question he asked me was that the chi-square result he was getting was very high.

He conducted a survey, and among other questions, one of them is a question which has 8 sub-questions about certain processes in company of examinees and the examinees have to give their grade of the quality of those processes by choosing values from 1 (bad) to 5 (great) (classical Likert scale).

There is one other value (sixth) that can be selected which is causing me trouble aside from the main problem. That sixth option is titled "N/A" and means that the examinees don't have that process in their company. Enough for intro (hopefully).

So the data table he made in Excel (for one subquestion) looks like this (there were 195 examinees in total):

Image 1.

I wonder why did he assume that theoretical frequencies (f$\_t$) are fixed to 32,5 (shown as rounded to 35 in data table) for all values? My opinions is that he is wrong, and that the theoretical frequencies should be these (and if not these, for sure not the same for every mark):

Image 2.

The chi-square value is even greater now. I am not even sure why does he calculate it in this fashion in Excel.

My opinion that for this data set chi-square value should be calculated with CHITEST function of the following syntax (cell addresses are as in Image 2.):

=CHITEST(C9:C14;D9:D14)

That way i get a chi-square value of 1,26177E-44, which I think is ok.

Please help me, all I thought I know or knew on topic seems to be vanished now. Sorry if any terms are not understandable, English is not my primary language. If any clarification is required, I'll do my best.

Thank you in advance for your time and knowledge and sorry that images are not automatically displayed, I need to gain more reputation in order to enable them.

$\endgroup$
  • 3
    $\begingroup$ Your difficulty is that you have yet to formulate a research question. What is the purpose of conducting this test? What do you hope to learn from the survey results? In the first image, you are merely testing whether the responses are uniformly distributed across all grades (including N/A!). Of course they are not, but so what? In the second image, it is not evident what the reference distribution is, but again the rejoinder is the same: so what? What does it tell you to conclude the responses do not have a particular distributional shape? Little or nothing, I would guess. $\endgroup$ – whuber Mar 6 '12 at 18:35
  • $\begingroup$ Thank you @whuber for you input. To be honest, I don't know as well what is my colleague hoping to learn from the survey results. I haven't even seen the survey questions, all he wanted to know is why (in his opinion) the chi-square values he was getting were so high. Frankly, I don't even need to know his goal, my dilemma is mainly the mechanism he is using to calculate the chi-square values. Is he wrong (setting all theoretical frequencies values to 33), or am I right (regarding values of theoretical frequencies and way of calculating chi-square values)? $\endgroup$ – user9645 Mar 6 '12 at 20:26
  • 1
    $\begingroup$ But you cannot know how to calculate the right chi-square values (or even whether to calculate chi-square at all) without knowing the goal. Solving statistical problems without context is like boxing blindfolded - you might knock your opponent out, but you might break your hand on the ringpost. $\endgroup$ – Peter Flom - Reinstate Monica Mar 6 '12 at 20:37
  • 1
    $\begingroup$ Ok, thanks for your input @PeterFlom. I'll contact my colleague tomorrow and ask him to tell me his defined goal(s). I'll also ask him to give me questions from survey and I'll translate them so then things will maybe get more clear. $\endgroup$ – user9645 Mar 6 '12 at 20:42
  • 1
    $\begingroup$ Peter's quite right, Branimir. We can still help a little: your colleague correctly computes a $\chi^2$ statistic for the purpose of assessing whether the responses are evenly distributed among the six categories. The statistic is very high because the responses are not evenly distributed. That answers your colleague's question. The concern evidenced in these comments is that in conducting this test, you and your colleague indicate you (probably) are not asking useful (or even meaningful) questions. Getting the question right is far more important than a correct calculation! $\endgroup$ – whuber Mar 6 '12 at 21:39
2
$\begingroup$

The first table is testing the assumption that all possible answers are equally likely, the 33 comes from $\frac{195}{6} = 32.5$, which then is being rounded to 33 for display. The testing looks correct after that (though as @whuber points out, this is probably a meaninless test to do).

In the second table you compute the "expected" values ($f_t$) in a different way, possibly from a normal distribution, but the numbers sum to 100 instead of 195. Your higher $\chi^2$ value is due to the fact that the columns add up to different numbers (195 vs. 100) and does not in any way fit in with the idea of a $\chi^2$ test. The results are fairly meaningless.

The value you report from the chitest looks more like it would be the p-value than the $\chi^2$ value, are you sure that you are not getting the 2 confused? That value is essentially 0 which would not be the test statistic for the differences in pattern seen there.

Figure out the question first, then look to find the test that answers that question.

$\endgroup$
  • $\begingroup$ Thank you @GregSnow, your valuable answer clarified my problems. I indeed confused p-value and chi-square value, probably due to simultaneous translation and copy-pasting. Would you be so kind to explain to me why is the test in first table meaningless? In second table I did use values from normal distribution, but failed to notice wrong sum of 100 instead of 195. $\endgroup$ – user9645 Mar 6 '12 at 22:14
  • 1
    $\begingroup$ Things are a bit clearer to me now. What could be in your opinion be some examples of questions that could be asked and appropriate tests? $\endgroup$ – user9645 Mar 6 '12 at 22:20
  • 2
    $\begingroup$ These sort of data are really only useful if you can compare to some kind of reference. For an example of a straight comparison, "is question 6.1 rated more positively than question 6.2?"; or (if you have similar surveys over a period of time) "Is the response to 6.1 this year better or worse than last year?" Second, you might be interested in relationships eg "do males answer this question more positively than females?" or "is a more positive question on 6.2 likely to lead to a more positive response to 6.1?" (but be careful, if the answer is yes, this can have lots of reasons). $\endgroup$ – Peter Ellis Mar 7 '12 at 3:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy