3
$\begingroup$

I am reading scientific research (Link) and stumbled upon the following notation: X²(1, N=100)=6.83, p =0.009. An explanation is given as follows:

    Thirty-four Thinspiration and 21 Fitspiration sites
    featured content related to losing fat or weight, 
    X^2(1, N=100)=6.83, p=0.009.
    Losing fat or weight: Thinspiration Percent=68%, Fitspiration Percent=42%

Except "X^2 analyses were conducted to compare website content", I can't find more information regarding this particular example.

  • What does N=100 mean? (I guess it stands for a total value, but how is it calculated, if we have 34+21=55 sites? N=55?)
  • What does 6.83 mean? (I guess it is some kind of average? But how is it calculated, based on the information provided?)
  • p=0.009 should be the p-value? And again, how is it calculated?

I am new to scientific research. I hope not to annoy anyone with my questions. I would be really delighted, if someone could help me out. What am I missing here?

Thank you so much

$\endgroup$
3
$\begingroup$

This notation is describing a statistical test that was applied in order to determine whether or not there is a "statistically significant" association between two variables. Here, the specific question is whether "content related to losing fat or weight" is associated with what site you are on (Thinsipration vs. Fitspiration). The null (default) hypothesis is that there is no difference in the frequency of this content between the two sites, and we're checking to see if there's evidence to refute that

To test the hypothesis, the authors examined 100 websites, which is the N, or sample size. Each of the 100 websites is marked as having the particular content or not. From this, we can generate a 2x2 contingency table that represents the co-occurence of the website type and the presence/absence of the particular content. From this table, we can compute the chi-squared statistic, which essentially compares the actual counts the you observe to the null hypothesis of just distributing counts randomly in the 2x2 table (details on the computation can be found here. The result is a chi-squared statistic, which is the value 6.83 that the authors mention. This value can be combined with the overall sample size (N) and "degrees of freedom" of the table (# rows-1 time # columns-1, here just 1) to determine the significance, or p-value, of the result (here, p=0.009). In all, the description given of X²(1, N=100)=6.83, p =0.009 indicates that the authors ran a chi-squared test with 1 degree of freedom on 100 samples, which yielded a test statistic of 6.83, which is unlikely to have been observed under the null hypothesis (p=0.009)

The p-value represents the likelihood that you would observe data as or more "extreme" than what you actually observed, under the null hypothesis of no association. The very low p-value here indicates that if Fitspiration and Thinspiration websites did indeed post "weight loss content" with equal frequency, it would extremely unlikely to see such a lopsided result when sampling 100 websites. There are 34 of 50 Thinspiration websites that post this content (68%), but only 21 of 50 Fitspiration websites that do so (42%). If both websites posted the content with the same frequency, it's possible that random sampling would get unlucky and find such a lopsided sample, but that would only happen 9 in 1000 times (p = 0.009). The chi-squared statistic, degrees of freedom, and the sample size combine to give you the p-value, since with more data, smaller differences become more meaningful - it would not be as surprising to find a 70%-40% imbalance if you only sampled 10 websites even if the content frequency was the same, but the observed imbalance becomes less and less probable as N grows. Similarly, if we increased the degrees of freedom (by testing the difference bewteen 5 websites instead of 2, for example), it's easier to find spurious random differences, since there are more opportunities to get "unlucky" with your sample selection

$\endgroup$
  • 2
    $\begingroup$ +1. I was able to reproduce the statistics with the R command chisq.test(matrix(c(34, 16, 21, 29), 2), correct=FALSE). Notice the lack of a continuity correction, which in this case lowers the p-value from 0.016 to 0.009. The higher p-value is the correct one, as demonstrated by a simulation with 1E7 iterations. This difference, although seemingly minor, could loom large when correcting for multiple testing. $\endgroup$ – whuber Jan 17 at 21:08
  • $\begingroup$ @whuber Great point. Also worth considering the possibility of using Fisher's exact test rather than a chi-squared test, which can be a bit conservative, and a bit harder to compute, but will work for arbitrarily small sample sizes (chi-squared approximations break down if the cell counts in the 2x2 table are too small, generally <5). The Fisher's test p-value here is 0.01542, which is quite close to the corrected chi-squared p-value of 0.01586. The Fisher test can be difficult to compute for very large N, but is often preferable if you are able to do so. $\endgroup$ – Nuclear Wang Jan 17 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.