0
$\begingroup$

I collected some data regarding purchase behaviour (purchasing a black or white car), before and after updating the background color of an e-commerce website.

I would like to know if the change was significant, so I applied the Chi Square statistic formula, and found a test statistic for the full table of 76.6 (it is the sum of the 4 cells = 14+13+25+25, red cell below).

However, when I look at the Chi square distribution table, with 1 degree of freedom, the test statistic 76.6 does not even appear on the table and I get stuck.

I read online about the Chi Square Goodness of Fit test.. by computing the 76.6, did I actually already made the fit test, without being aware of it? Shall I conclude that 76.6 is a value too large, and Chi Square is not the right model to fit my data?

Please find below also the data with observed, expected and test statistic. Thanks!

enter image description here

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ With a test statistic of 76, then it is an extremely rare chance event. You can safely reject the null hypothesis. $\endgroup$
    – Dave2e
    Jan 25, 2021 at 3:31
  • $\begingroup$ The assumption of a chi-squared test is that the numbers are following a Poisson distribution and 'mean = variance'. With ~2500 items you get a standard deviation of around √2500 = 50. But it might be that the sales are not like a Poisson process. One example is when the sales are with multiple items at a time. For cars at an e-commerce site this might not be the case but there could be other reasons for larger fluctuations. Like, for instance, holidays or other events, causing fluctuations in sales much larger than the standard deviation of 50 (and which may effect the colors differently). $\endgroup$
    – Sextus Empiricus
    Nov 6, 2021 at 8:35
  • $\begingroup$ So ideally you have sales over a longer period based on which you can estimate the variation that randomly occurs and how it can be different for white and black even without background update of the website. $\endgroup$
    – Sextus Empiricus
    Nov 6, 2021 at 8:37

2 Answers 2

Reset to default
0
$\begingroup$

@BruceET's has already answered but since you are asking about drawing conclusions, I would like to add a couple of points.

With many counts - in the order of thousands - many real-world problems would give a very small pvalue. However, I wouldn't confuse a small pvalue for an effect that must necessarily be important. In your case the CW update seems to affect white cars more than black cars, the percent changes being

$$ Change_{white} = log(2810) - log(1229) = 83\% \\ Change_{black} = log(2402) - log(1585) = 42\% $$

However, this difference may or may not be practically relevant. Also, if you had 100 times fewer counts (like: 28, 12, 24, 12) you would have a non-significant p-value even if the effect size stays the same.

The other consideration is that the $\chi^2$ test assumes counts coming from purely random draws like balls from a bag. In practice, there is often additional variation between different categories so you may get a large differences even if nothing interesting is actually happening.

Hope this hekos and I got things right!

Improve this answer
$\endgroup$
0
$\begingroup$

Your question is quite unclear. But if you are doing a chi-squared test of independence on the table TAB below, results from chisq.test are as below.

The very large chi-squared statistic $76.64$ and correspondingly tiny P-value (nearly $0)$ are shown. [Because of the large counts, I did not use Yates' continuity correction).]

    TAB = rbind(c(2402, 1585), c(2810, 1229))
    TAB
          [,1] [,2]
     [1,] 2402 1585
     [2,] 2810 1229

    chisq.test(TAB, cor=FALSE)

            Pearson's Chi-squared test

    data:  TAB
    X-squared = 76.643, df = 1, p-value < 2.2e-16

A chi-squared statistic larger than $3.8415$ would have led to rejection at the 5% level (also $6.6349$ at the 1% level; $10.8276$ at the 0.1% level).

    qchisq(c(.95,.99,.999), 1)
    [1]  3.841459  6.634897 10.827566

So your chi-squared statistic $76.634$ is too large to be found in a printed table, but leads to rejection at a very small level of significance.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.