2
$\begingroup$

I quote RACHEV, HÖCHSTÖTTER, FABOZZI, FOCARDI (2010).

Starting from the bivariate variable $(x,y)$, with the component variable $x$ taking values in $(v_i, i=1,\dots,r)$ and the component variable $y$ taking values in $(w_j, j=1,\dots,s)$, one can define the chi-square test statistic as: \begin{equation} \chi^2=n\sum_{i=1}^r\sum_{j=1}^s\frac{\left(f_{x,y}\left(v_i, w_j\right)-f_x\left(v_i\right)f_y\left(w_j\right)\right)^2}{f_x\left(v_i\right)f_y\left(w_j\right)}\tag{1} \end{equation} with $f_{x,y}$ denoting the relative frequency of the bivariate variable $(x,y)$, $f_x$ denoting the relative frequency of the variable $x$ and $f_y$ denoting the relative frequency of the variable $y$.

One problem arises with $\chi^2$ as to its dependence on the data size $n$. For increasing $n$, the statistic can grow beyond any bound such that there is no theoretical maximum. The solution to this problem is given by the so-called Pearson contingency coefficient (or simply contingency coefficient) defined by: \begin{equation} C=\sqrt{\frac{\chi^2}{\chi^2+n}}\tag{2} \end{equation} Clearly, $C$ is such that $0\leq C< 1$. Consequently, it assumes values which are strictly less than one but may become arbitrarily close to one. $\color{red}{\textrm{This is still not satisfactory enough for our purpose to design a measure that can}}$ $\color{red}{\textrm{uniquely determine}}$ $\color{red}{\textrm{the respective degrees of dependence of different data sets.}}$

There is another coefficient that can be based on the following. In the extreme case of total dependence of $x$ and $y$, each variable will assume a certain value if and only if the other variable assumes a particular corresponding value. Hence, we have $k=\min\left\{r,s\right\}$ unique pairs that occur with positive frequency. $\color{red}{\textrm{Then one can show that:}}$ \begin{equation} \color{red}{C=\sqrt{\frac{k-1}{k}}\tag{3}} \end{equation} such that, generally, $0\leq C\leq \sqrt{\frac{k-1}{k}}<1$. $\color{red}{\textrm{Now, the standardized coefficient can be given by:}}$ \begin{equation} \color{red}{C_{\text{corr}}=\sqrt{\frac{k}{k-1}}C\tag{4}} \end{equation} which is called the corrected contingency coefficient with $0\leq C\leq 1$.



Could you please help me understand the $\color{red}{\textrm{three parts in red above?}}$. In general, which is the "aim" pursued in passing from $(2)$ to $\color{red}{(4)}$?

$\endgroup$

1 Answer 1

1
+100
$\begingroup$

The Pearson contingency coefficient is intended to provide us some measure regarding the dependence between data columns (which sometimes, when discussing regression, we call collinearity). $C$ takes values in $[0,1)$ when values near 0 indicate column independence, values far from 0 indicate dependence. But how far is far? Can we achieve a tighter upper bound? That's the general idea.

Red part (a): notice the term "degrees of dependence". If there are zero of them, columns are independent.

Red part (c): as the upper bound for $C$ was found, dividing by this bound (/multiplying by its reciprocal) $C_{corr}=\sqrt{\frac{k}{k-1}}C$ yields $0\le C_{corr} \le 1$.

Red part (b): Let $k=\min\{r,s\}$ and assume the extreme condition described (e.g. $x,y$ is a pair of letter and digit. Although there are 10 digits and 26 letters, The only pairs which appear are $(a,0), (s,1), (h,2), (i,3), (r,4), (g,5), (u,6), (n,7), (y,8), (t,9)$). Each of the $k=10$ possible pairs has relative frequency $f_{xy}$, as well as each of the components (i.e each letter and digit have frequencies $f_x,f_y$).

For letters outside the $k$ used above, $f_x=0$ and obviously $f_{xy}=0$ so they're not summed. Our sum is now: $$\chi^2=n\sum_{i=1}^k\sum_{j=1}^k\frac{(f_{x,y}(v_i, w_j)-f_x(v_i)f_y(w_j))^2}{f_x\left(v_i\right)f_y(w_j)}\\=n\sum_{i=1}^k\sum_{j=1}^k\left[\frac{f^2_{xy}(v_i,w_j)}{f_x(v_i)f_y(w_j)}\right]-2n\sum_{i=1}^k\sum_{j=1}^k\left[f_{xy}(v_i,w_j)\right]+n\sum_{i=1}^k\sum_{j=1}^k\left[f_x(v_i)f_y(w_j)\right]$$

The rightmost sum equals 1 as well as the middle sum (second probability axiom), so we only need to solve

$$\chi^2=n\sum_{i=1}^k\sum_{j=1}^k\left[\frac{f^2_{xy}(v_i,w_j)}{f_x(v_i)f_y(w_j)}\right]-2n+n=n\sum_{i=1}^k\sum_{j=1}^k\left[\frac{f^2_{xy}(v_i,w_j)}{f_x(v_i)f_y(w_j)}\right]-n$$

If the pair $(v_i,w_j)$ does not appear on our list, $f_{xy}(v_i,w_j)=0$ is not summed. The pair matching is one-to-one (a digit always appears with the same letter and vice versa), so now assume that a pair $(v_i,w_j)$ does exist. In fact, it appears $a$ times (out of $n$), so its relative pair frequency is $f_{xy}(v_i,w_j)=\frac{a}{n}$. However, these are also the only $a$ occurrences of $v_i$ and $w_j$ so $f_x(v_i)=f_y(w_j)=f_{xy}(v_i,w_j)=\frac{a}{n}$. This means that when a pair does exist, the fraction equals $1$. We have $k$ such cases, so overall $$\chi^2=nk-n=n(k-1)$$

Now getting the boundary for $C$ is easy:

$$C=\sqrt{\frac{\chi^2}{\chi^2+n}}=\sqrt{\frac{n(k-1)}{n(k-1)+n}}=\sqrt{\frac{k-1}{k}} \qquad\blacksquare.$$

$\endgroup$
2
  • 1
    $\begingroup$ First of all, thank you a lot. Just three questions: 1) Above, you meant "The rightmost sum equals 1" (instead of "The leftmost sum equals 1"), didn't you? 2) I cannot see why $\sum_{i=1}^k\sum_{j=1}^k\left[f_x(v_i)f_y(w_j)\right]$ can be proven to be equal to $1$ thanks to the second probability axiom. Could you please give me an insight? 3) Why does $C_{\text{corr}}=\sqrt{\frac{k}{k-1}}C$ yield that $0\leq C_{\text{corr}}\leq1$? $\endgroup$ Oct 20, 2021 at 8:29
  • 1
    $\begingroup$ 1) Thanks, corrected it. 2) The sum of $f_y(w_j)$ over all $j$ values is 1. That's the axiom. The sum $\sum_{j=1}^{k}{f_x(v_i)f_y(w_j)}$ is thus $f_x(v_i)$. The sum of $f_x(v_i)$ over all $i$ values is 1. That's the axiom again. We therefore get that $\sum_{i=1}^k\sum_{j=1}^k\left[f_x(v_i)f_y(w_j)\right]=1$. That's (also) the 2D integral for the product of 2 marginal PDFs (which is the axiom once again). 3) If $\max(C)=\sqrt{\frac{k-1}{k}}$ then $\max(C_{corr})=\sqrt{\frac{k}{k-1}}\cdot\max(C)=\sqrt{\frac{k}{k-1}}\cdot\sqrt{\frac{k-1}{k}}=1$. $\endgroup$
    – Spätzle
    Oct 20, 2021 at 9:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.