The very limited information you have is certainly a severe constraint! However, things aren't entirely hopeless.
Under the same assumptions that lead to the asymptotic $\chi^2$ distribution for the test statistic of the goodness-of-fit test of the same name, the test statistic under the alternative hypothesis has, asymptotically, a noncentral $\chi^2$ distribution. If we assume the two stimuli are a) significant, and b) have the same effect, the associated test statistics will have the same asymptotic noncentral $\chi^2$ distribution. We can use this to construct a test - basically, by estimating the noncentrality parameter $\lambda$ and seeing whether the test statistics are far in the tails of the noncentral $\chi^2(18, \hat{\lambda}) $ distribution. (That's not to say this test will have much power, though.)
We can estimate the noncentrality parameter given the two test statistics by taking their average and subtracting the degrees of freedom (a methods of moments estimator), giving an estimate of 44, or by maximum likelihood:
x <- c(45, 79)
n <- 18
ll <- function(ncp, n, x) sum(dchisq(x, n, ncp, log=TRUE))
foo <- optimize(ll, c(30,60), n=n, x=x, maximum=TRUE)
> foo$maximum
[1] 43.67619
Good agreement between our two estimates, not actually surprising given two data points and the 18 degrees of freedom. Now to calculate a p-value:
> pchisq(x, n, foo$maximum)
[1] 0.1190264 0.8798421
So our p-value is 0.12, not sufficient to reject the null hypothesis that the two stimuli are the same.
Does this test actually have (roughly) a 5% reject rate when the noncentrality parameters are the same? Does it have any power? We'll attempt to answer these questions by constructing a power curve as follows. First, we fix the average $\lambda$ at the estimated value of 43.68. The alternative distributions for the two test statistics will be noncentral $\chi^2$ with 18 degrees of freedom and noncentrality parameters $(\lambda-\delta, \lambda+\delta)$ for $\delta = 1, 2, \dots, 15$. We'll simulate 10000 draws from these two distributions for each $\delta$ and see how often our test rejects at, say, the 90% and 95% level of confidence.
nreject05 <- nreject10 <- rep(0,16)
delta <- 0:15
lambda <- foo$maximum
for (d in delta)
{
for (i in 1:10000)
{
x <- rchisq(2, n, ncp=c(lambda+d,lambda-d))
lhat <- optimize(ll, c(5,95), n=n, x=x, maximum=TRUE)$maximum
pval <- pchisq(min(x), n, lhat)
nreject05[d+1] <- nreject05[d+1] + (pval < 0.05)
nreject10[d+1] <- nreject10[d+1] + (pval < 0.10)
}
}
preject05 <- nreject05 / 10000
preject10 <- nreject10 / 10000
plot(preject05~delta, type='l', lty=1, lwd=2,
ylim = c(0, 0.4),
xlab = "1/2 difference between NCPs",
ylab = "Simulated rejection rates",
main = "")
lines(preject10~delta, type='l', lty=2, lwd=2)
legend("topleft",legend=c(expression(paste(alpha, " = 0.05")),
expression(paste(alpha, " = 0.10"))),
lty=c(1,2), lwd=2)
which gives the following:
Looking at the true null hypothesis points (x-axis value = 0), we see that the test is conservative, in that it doesn't appear to reject as often as the level would indicate, but not overwhelmingly so. As we expected, it doesn't have much power, but it's better than nothing. I wonder if there are better tests out there, given the very limited amount of information you have available.
