0
$\begingroup$

Take this example:

data <-matrix(c(227,751,193,541), ncol=2)
column1 <- c(227, 751)
probabilities <- c( 193/(193+541), 541/(193+541) )

chisq.test(data)
chisq.test(column1, p= probabilities)

when i apply the chi-squared test providing a matrix the results says that this is a

Pearson's Chi-squared test with Yates' continuity correction

and provides a p-value of 0.158.

when i perform the second chi-squared, providing the first column of the matrix and the probabilities calculated from the second column the both the results and the name of the test change dramatically:

Chi-squared test for given probabilities

the reported p-value is 0.028.

considering that i am trying to determine if the two datasets i have (columns in the matrix) are NOT different from each other:

what is the difference between these two tests? which one should i use?

$\endgroup$
3
$\begingroup$

You're running two completely different tests. The first is a (two variable) test of no association in a 2x2 contingency table. The second is of a (one variable) test that the observed proportion is equal to the provided frequency.

Which one you should use depends on what exactly you are measuring.

  • Are you measuring on one variable and trying to compare it to the proportions in a reference population?
    Example: measuring the # of successes/failures, and comparing it to a reference set
  • Are you measuring on two variables and trying to determine if there is an association between them?
    Example: measuring the # of successes/failures by blue and red, and seeing if there is an association of success/failure with blue/red

I suspect it's the latter you're trying to do, and thus should use the first test.


To answer your comment - see this answer for how the (by-hand) calculation differs.

And as for why specifying the proportions matters (and is different from giving both replications) is that you are making an assumption that the proportion you give are the "true" value. Why is one replication the expected value and not the other?

data <-matrix(c(227,751,193,541), ncol=2)
c1 <- data[,1]
p1 <- data[,2] / sum(data[,2])
c2 <- data[,2]
p2 <- data[,1] / sum(data[,1])

[1] chisq.test(c1, p=p1)
[2] chisq.test(c2, p=p2)

Why do you perform [1] when [2] is just as valid and gives you a different result? (Though in this case, the conclusion is the same if you are looking at a cutoff value)

If that assumption that one measurement is the "real" proportion that should be compared to - fine, the second test will do. But you generally can't justify that.

$\endgroup$
  • $\begingroup$ I am trying to determine that the two samples I have (the two columns in the matrix) are not significantly different between them. what i do not understand is the difference between comparing two variable and comparing one variable to reference frequencies (which are calculated from the second variable!) $\endgroup$ – Tito Candelli Apr 24 '14 at 15:08
  • $\begingroup$ Adding to Affine's answer (+1 to Affine for that): You want the first test. The second test would be for something like seeing if a die roll was coming up 1/6 for each number, or if a coin was coming up half heads and half tails. $\endgroup$ – Peter Flom - Reinstate Monica Apr 24 '14 at 17:20
  • $\begingroup$ Tito, this information should be in your question. It sounds like you're talking about a test of homogeneity, and as Peter says, for that you want to use the first one, not the second. $\endgroup$ – Glen_b -Reinstate Monica Apr 24 '14 at 20:46
  • $\begingroup$ i will edit the question to reflect the comment. so to what determines the difference in p-value due? what are the mathematical workings of the two tests? what i really do not understand is the difference between calculating the probabilities from the reference set beforehand and supplying the probabilities to the test, or directly provide the data to the test. does the size of the dataset play a role? $\endgroup$ – Tito Candelli Apr 25 '14 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.