Meaning of chi-squared in R Kruskal-Wallis test

Question

I have a simple question to ask, I think it was not covered in other questions.

I am running a Kruskal-Wallis test in R, using the function kruskal.test. The output gives me the p-value and a chi-squared value (see example below)

Kruskal-Wallis rank sum test

data: mat[, 2] by mat[, 3]

Kruskal-Wallis chi-squared = 0.052043, df = 1, p-value = 0.8195

I know that I have to look at the p-value to know if there is a significant difference among the groups I'm testing, but I am interested in the chi-squared. I noticed that the chi-square increases when the p-value decreases, but its statistical meaning is not yet clear to me. There is not an explanation of its meaning in ?kruskal.test.

Can someone help me understand this?

Answer 1

The Kruskal Wallis chi-square statistic

The "Kruskal Wallis chi-squared" value reported by the R function is equal to the statistic $H$ that is computed in the test. If there are no ties then

$$H = \frac {N-1}{N}\sum_{i=1}^C \frac {\left(\bar {R_i}-\bar {R}\right)^2}{(N^2-1)/12}$$

where $\bar{R_i}$ is the mean of the ranks in the $i$-th sample and $\bar{R}=\frac{1}{2}(N+1)$ is the mean of all ranks.

It is named like this because the statistic follows approximately a chi squared distribution. Under the hood you can see it as the means $\bar {R_i}-\bar{R}$ being approximated as normal distributions with variance $\frac{1}{12}(N^2-1)$.

See:

Kruskal, William H., and W. Allen Wallis. "Use of ranks in one-criterion variance analysis." Journal of the American statistical Association 47.260 (1952): 583-621. https://doi.org/10.1080/01621459.1952.10483441

...computing a statistic $H$. Under the stated hypothesis, $H$ is distributed approximately as $\chi^2(C – 1)$...

The p-value

For the Kruskal Wallis test the p-value is $P(H_{\text{if $H_0$ true}} \geq H_{\text{observed}})$, a way to indicate how extreme a particular measurement $H_{\text{observed}}$ is by stating the probabilty that the value for an experiment when the null hypothesis is true, $H_{\text{if $H_0$ true}}$, would be equal or higher.

If the null hypothesis is false then you will be more likely to get such high/extreme values, thus when you observe an unlikely (ie low p-value) extreme value $H$ this indicates that the null/no-effect hypothesis may be false or at least is not supported by the data.

Exploring R functions

Whenever you have problems with R it can be helpful to look into the source code. This is fairly easy for most functions you just type the function name into the console and the source code is printed. Some functions are hidden and then you can use this

The source-code:

note the STATISTIC value at the end.

> getAnywhere(kruskal.test.default)
A single object matching ‘kruskal.test.default’ was found
It was found in the following places
  registered S3 method for kruskal.test from namespace stats
  namespace:stats
with value

function (x, g, ...) 
{
    if (is.list(x)) {
        if (length(x) < 2L) 
            stop("'x' must be a list with at least 2 elements")
        if (!missing(g)) 
            warning("'x' is a list, so ignoring argument 'g'")
        DNAME <- deparse(substitute(x))
        x <- lapply(x, function(u) u <- u[complete.cases(u)])
        if (!all(sapply(x, is.numeric))) 
            warning("some elements of 'x' are not numeric and will be coerced to numeric")
        k <- length(x)
        l <- sapply(x, "length")
        if (any(l == 0L)) 
            stop("all groups must contain data")
        g <- factor(rep.int(seq_len(k), l))
        x <- unlist(x)
    }
    else {
        if (length(x) != length(g)) 
            stop("'x' and 'g' must have the same length")
        DNAME <- paste(deparse(substitute(x)), "and", deparse(substitute(g)))
        OK <- complete.cases(x, g)
        x <- x[OK]
        g <- g[OK]
        if (!all(is.finite(g))) 
            stop("all group levels must be finite")
        g <- factor(g)
        k <- nlevels(g)
        if (k < 2L) 
            stop("all observations are in the same group")
    }
    n <- length(x)
    if (n < 2L) 
        stop("not enough observations")
    r <- rank(x)
    TIES <- table(x)
    STATISTIC <- sum(tapply(r, g, "sum")^2/tapply(r, g, "length"))
    STATISTIC <- ((12 * STATISTIC/(n * (n + 1)) - 3 * (n + 1))/(1 - 
        sum(TIES^3 - TIES)/(n^3 - n)))
    PARAMETER <- k - 1L
    PVAL <- pchisq(STATISTIC, PARAMETER, lower.tail = FALSE)
    names(STATISTIC) <- "Kruskal-Wallis chi-squared"
    names(PARAMETER) <- "df"
    RVAL <- list(statistic = STATISTIC, parameter = PARAMETER, 
        p.value = PVAL, method = "Kruskal-Wallis rank sum test", 
        data.name = DNAME)
    class(RVAL) <- "htest"
    return(RVAL)
}
<bytecode: 0x33a2ffb0>
<environment: namespace:stats>
> 

Answer 2

A chi-square statistic is the sum of the squared deviations for some expected pattern. If there are minimal deviations, then the chi-squared is small and the p-value is "chance-like", i.e. it's not small enough to be considered evidence of "significant" deviations from chance. In your case the rankings of paired observations are being calculated, but there are many statistical tests that use the chi-squared statistic that test a wide variety of data situations. So it's important to understand how each such statistic is constructed in order to understand what patterns might be chance like versus being "significant".

It is generally true that increases in the chi-squared statistic are associated with decreases in the reported p-value. For 1 degree of freedom the critical value (where p-value crosses the magic line of 0.05) of $\chi^2_1$ will always be 3.84, just as the critical value of Normal-theory tests is 1.96. Notice that 1.96^2 equals 3.84. That is not a chance-like finding.

You might notice in the code copied in another answer that the STATISTIC is formed by first squaring ranks within groups and then adding them together:

STATISTIC <- sum(tapply(r, g, "sum")^2/tapply(r, g, "length"))

The group contribution get divided by their lengths and then there is some further adjustment so that the final STATISTIC value will be on the same scale as the $\chi^2(k – 1)$, the probability distribution with degrees of freedom equal to the number of groups minus 1. The p-value is then calculated from the STATISTIC's location in that distribution

STATISTIC <- ((12 * STATISTIC/(n * (n + 1)) - 3 * (n + 1))/(1 - 
    sum(TIES^3 - TIES)/(n^3 - n)))
PARAMETER <- k - 1L
PVAL <- pchisq(STATISTIC, PARAMETER, lower.tail = FALSE)

Answer 3

If we take the p-value to represent the likelihood that the associated test statistic would be found if the Null hypothesis (that there are no differences among groups) is true, it make sense that large test statistics (e.g., the chi-square value in the kruskal-wallis) would be associated with small p-values (i.e. lessened likelihood). Larger chi-square values are found at the tail end of the chi-square distribution, which are associated with smaller probabilities.

So, what you're noticing when you say that the "chi-square increases when the p-value decreases" is that larger chi-square values are more rare under the Null hypothesis.