Can I use specific expected values in a chi-squared test?

Question

EDIT: More context in experiment, hypotheses, and more clarity in my questions. TL;DR below.

Fifty participants completed two similar tasks under three conditions:

• No distraction
• Low distraction
• High distraction

A theory suggests both tasks measure the same mechanism. In addition, the theory says performance on both tasks will be highest in the no distraction condition, followed by the low distraction condition, with lowest performance in the high distraction condition.

I have already tested the following predictions on the continuous scores:

• Performance on no dist. > lo dist. > hi dist. in Task 1 [TRUE]
• Performance on no dist. > lo dist. > hi dist. in Task 2 [TRUE]
• Performance on the two tasks will be correlated for each condition [FALSE]

Since the third prediction was incorrect using the continuous scores, I've decided to take an additional approach and reduce this data down to frequencies.

These are my observed values for 'best' scores:

Best Score           Best Score in Task 1
in Task 2      No Dist.      Lo Dist.      Hi Dist.
No Dist.          12            5             1    
Lo Dist.          14            8             2
Hi Dist.           4            3             1

And my observed values for 'worst' scores:

Worst Score           Worst Score in Task 1
in Task 2      No Dist.      Lo Dist.      Hi Dist.
No Dist.           1            2             4    
Lo Dist.           1            2             5
Hi Dist.           6            4            25

And mosaic plots for these values, where tile area relates to participant frequency:

I'd like to test the hypotheses:

• More participants will achieve their best scores in both tasks in the 'no distraction' condition, than in the 'low distraction' condition; (1,1) > (2,2)
• Less participants will achieve their best scores in both tasks in the 'high distraction' condition, than in the 'low distraction' condition; (3,3) < (2,2)
• More participants will achieve their worst scores in both tasks in the 'high distraction' condition, than in the 'low distraction' condition; (3,3) > (2,2)
• Less participants will achieve their worst scores in both tasks in the 'no distraction' condition, than in the 'low distraction' condition; (1,1) < (2,2)

These hypotheses can be visualised with the mosaic plots below (n.b. arbitrary values).

A classic chi-squared test of independence would not answer my specific hypotheses. In fact they suggest best/worst scores on the two tasks to be independent.

If the exact values from these plots are used, my expected values for best scores would be:

Best Score           Best Score in Task 1
in Task 2      No Dist.      Lo Dist.      Hi Dist.
No Dist.         12.5          8.3          4.2    
Lo Dist.          8.3          5.5          2.8
Hi Dist.          4.2          2.8          1.4

And for worst scores would be the inverse:

Worst Score           Worst Score in Task 1
in Task 2      No Dist.      Lo Dist.      Hi Dist.
No Dist.          1.4          2.8          4.2    
Lo Dist.          2.8          5.5          8.3
Hi Dist.          4.2          8.3         12.5

However, the magnitude of the frequencies in these plots are arbitrary. It's the relative proportions between conditions that are important. Therefore, testing these specific expected values may not be fair.

My questions: (TL;DR)

1. Can I use specific expected values in a chi-squared test?
   • If so, would it be fair to use the arbitrary values in the latter two tables?
2. Should I test against strong and weak versions of the hypothesis, in terms of expected values?
   • If so, could I compare their fits somehow? e.g. a likelihood ratio test

Answer 1

I would relax your null hypothesis a bit. I would say that there are people who switch condition, and when they do they will follow an independence model (what you test with a standard $\chi^2$-test), but that there is also a substantial portion of the respondents that remain on the diagonals. This is a classic hypothesis in log-linear analysis.

You test that by estimating two models: one that is only an independence model, and one that adds parameters for the diagonals. The exponentiated diagonal parameters are the factors by which the counts in the diagonal cells increase due to the people who choose to stay on the diagonal cells (or if they are smaller than 1, choose to avoid the diagonal cells): the expected number of people in the diagonal cell 1,1 increases by a factor 1.5 compared to an independence model due the people that per se want to remain in a diagonal cell. Then we compare the two models with a likelihood ratio test. In your data we canot reject the hypothesis that there is nothing special about the diagonal.

Here is how I would do this in Stata:

. // prepare the data
. clear

. input task1 task2 freq

         task1      task2       freq
  1. 1 1 12
  2. 1 2  5
  3. 1 3  1
  4. 2 1  4
  5. 2 2  3
  6. 2 3  1
  7. 3 1 14
  8. 3 2  8
  9. 3 3  2
 10. end

.
. gen diag = (task1==task2)*task1

.
. // look at the data
. tabdisp task1 task2, c(freq)

----------------------------
          |      task2
    task1 |    1     2     3
----------+-----------------
        1 |   12     5     1
        2 |    4     3     1
        3 |   14     8     2
----------------------------

. tabdisp task1 task2, c(diag)

----------------------------
          |      task2
    task1 |    1     2     3
----------+-----------------
        1 |    1     0     0
        2 |    0     2     0
        3 |    0     0     3
----------------------------

.
. // test whether diagonals are more common than
. // we would expect under an independence model
.
. // the independence model
. poisson freq i.task1 i.task2, irr

Iteration 0:   log likelihood = -14.980851
Iteration 1:   log likelihood =  -14.95665
Iteration 2:   log likelihood = -14.956607
Iteration 3:   log likelihood = -14.956607

Poisson regression                              Number of obs     =          9
                                                LR chi2(4)        =      31.07
                                                Prob > chi2       =     0.0000
Log likelihood = -14.956607                     Pseudo R2         =     0.5095

------------------------------------------------------------------------------
        freq |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       task1 |
          2  |   .4444444   .1888526    -1.91   0.056     .1932531    1.022136
          3  |   1.333333   .4157397     0.92   0.356     .7236524    2.456674
             |
       task2 |
          2  |   .5333333   .1651038    -2.03   0.042     .2907319    .9783739
          3  |   .1333333   .0709721    -3.79   0.000     .0469734    .3784647
             |
       _cons |       10.8   2.834643     9.07   0.000     6.456713    18.06492
------------------------------------------------------------------------------

. est store independence

.
. // allow diagonals to be different
. poisson freq i.task1 i.task2 i.diag, irr

Iteration 0:   log likelihood = -14.724323
Iteration 1:   log likelihood = -14.672566
Iteration 2:   log likelihood = -14.672395
Iteration 3:   log likelihood = -14.672395

Poisson regression                              Number of obs     =          9
                                                LR chi2(7)        =      31.64
                                                Prob > chi2       =     0.0000
Log likelihood = -14.672395                     Pseudo R2         =     0.5188

------------------------------------------------------------------------------
        freq |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       task1 |
          2  |   .5506669   .3871607    -0.85   0.396     .1388128    2.184481
          3  |   1.759949   .9537229     1.04   0.297     .6084581    5.090604
             |
       task2 |
          2  |   .6046409    .256736    -1.18   0.236      .263069    1.389714
          3  |   .1655643    .137213    -2.17   0.030     .0326231    .8402489
             |
        diag |
          1  |    1.54041   1.011098     0.66   0.510     .4255227    5.576352
          2  |   1.156618   .9645865     0.17   0.861      .225586    5.930179
          3  |   .8810864   .9542909    -0.12   0.907     .1054618    7.361081
             |
       _cons |   7.790132   4.592245     3.48   0.000     2.453384    24.73569
------------------------------------------------------------------------------

. est store diag

.
. // test the two models agains one another
. lrtest diag independence

Likelihood-ratio test                                 LR chi2(3)  =      0.57
(Assumption: independence nested in diag)             Prob > chi2 =    0.9036

Answer 2

You can test that hypothesis directly with a chi-square goodness of fit. Each cell of your design is a cell with a hypothesized proportion (this is what the test of independence is doing anyway, it just is coming up with a way of combining the marginals to come up with the appropriate proportions assuming independence).

Using raw counts...

$ \chi^2 = \Sigma\frac{(O - E)^2}{E} $

Where O is your observed count under the null and E is your expected count. Although it should be noted that E will fall apart under expectations of 0 (and generally chi-square doesn't perform well with low count cells... so some allowances may need to be made... a hypothesized proportion of 0 is a pretty extreme hypothesis).