It is said that if the plots of the hypothetical responses are not parallel, but crossed, there is interaction. Suppose we have two factors. Is it possible that the plots cross but we do not have interaction? That is more reasonable when the plots are close to each other.
I noticed the converse is true. We may have interaction even if neither the curves of factor A on factor B nor factor B on factor A intersect. In a book I read this happens when the interaction is removable, which means that there is another cognate independent variable.
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
This depends on what is meant by "interaction". If the data have no noise - the plot is literally just two parallel lines, then there is certainly no interaction, we know this deductively, without any need for statistics. Secondly if the lines are not parallel, then we know deductively that there is interaction. So there is no counter example if there is no noise.
But if there is noise (or error), then there is basically more than one possible place that the "noiseless" or "true" lines could be. It is also possible for the true lines to be parallel but if the noise is big enough and you get an "unlucky" sample of noise, then the noisy lines will cross. Just how unlucky depends on how "non-parallel" the two "true lines" are and how many units have been sampled. Consider the OLS case, the lines are generated by:
$$y_{i}=x_{i}^{T}\beta_{true}+n_{i}$$
Where $\beta_{true}$ is a 4-D vector with the intercept for group 1, the offset for group 2, the slope for group 1 and the offset to the slope for group 2
Now you fit an OLS to the observed data, and you get
$$\beta_{OLS}=(X^{T}X)^{-1}X^{T}Y=(X^{T}X)^{-1}X^{T}(X\beta_{true}+n)=\beta_{true}+(X^{T}X)^{-1}X^{T}n$$
So by a careful choice of the noise we can make the OLS estimates be basically anything. So I don't have to invert a $4\times 4$ matrix, I will specialise to the case where both intercepts are equal to zero, and we have
$$y_{ij}=\beta_{1}x_{ij}+\beta_{2}x_{i2}I(j=2)$$
And then $$(X^{T}X)^{-1}=\frac{1}{\left(\sum_{i}x_{i1}^{2}\right)\left(\sum_{i}x_{i2}^{2}\right)}\begin{pmatrix} \sum_{i}x_{i1}^{2}+\sum_{i}x_{i2}^{2} & -\sum_{i}x_{i2}^{2} \\ -\sum_{i}x_{i2}^{2} & \sum_{i}x_{i2}^{2} \end{pmatrix}$$ $$=\frac{1}{\sum_{i}x_{i1}^{2}}\begin{pmatrix} 1 & -1 \\ -1 & 1\end{pmatrix} +\frac{1}{\sum_{i}x_{i2}^{2}}\begin{pmatrix} 1 & 0 \\ 0 & 0\end{pmatrix}$$
Now for $X^{T}n$ we have:
$$X^{T}n=\sum_{i}x_{i2}n_{i2}\begin{pmatrix} 1\\1\end{pmatrix} +\sum_{i}x_{i1}n_{i1}\begin{pmatrix} 1\\0\end{pmatrix}$$
And so the total error from the regression is:
$$\frac{\sum_{i}x_{i2}n_{i2}+\sum_{i}x_{i1}n_{i1}}{\sum_{i}x_{i2}^{2}}\begin{pmatrix} 1\\0\end{pmatrix} +\frac{\sum_{i}x_{i1}n_{i1}}{\sum_{i}x_{i1}^{2}}\begin{pmatrix} 1\\-1\end{pmatrix}$$
Now if the true slopes are parallel, so that $\beta_{2,true}=0$, then the OLS estimates will be:
$$\hat{\beta}_{1}=\beta_{1,true}+\frac{\sum_{i}x_{i2}n_{i2}+\sum_{i}x_{i1}n_{i1}}{\sum_{i}x_{i2}^{2}}+\frac{\sum_{i}x_{i1}n_{i1}}{\sum_{i}x_{i1}^{2}}$$ $$\hat{\beta}_{2}=-\frac{\sum_{i}x_{i1}n_{i1}}{\sum_{i}x_{i1}^{2}}$$
Now this shows that the OLS estimate can indeed lead to erroneous interactions, just choose the "true" noise such that it is highly correlated with $x_{i1}$ - essentially you need to violate one of the assumptions of OLS, non-heteroscedasticity of the noise. So if you generate data according to:
$$y_{i1}=x_{i1}(\beta_{1,true}+n_{i1})$$ $$y_{i2}=x_{i2}\beta_{1,true}+n_{i2}$$
And then try to fit an interaction model using OLS of $y$ on $x$ with an interaction, you will find a significant result, even though the true betas are the same. The plots will cross because of the fanning in the first group.
One example data set (true beta is 2 and noise was generated from standard normal). You get a t-statistic above 10 for the interaction effect:
$$\begin{array}{c|c} group & y & x \\ 1 & 1.282817715 & 1 \\ 1 & 2.026032115 & 2 \\ 1 & 5.9786882 & 3 \\ 1 & 22.1588319 & 7 \\ 2 & 16.28587668 & 9 \\ 2 & 15.12007527 & 6 \\ 2 & 9.566273403 & 5 \\ \end{array}$$
answered May 1, 2011 at 9:55
$\begingroup$
$\endgroup$
To me it seems like you (and many books probably) are confusing the empirical level with the theoretical level: The null hypothesis of an interaction effect in a two-way ANOVA is defined on the theoretical level using the cell expected values $\mu_{jk}$ (and not response values): there is an interaction if (and only if) the lines connecting the $\mu_{jk}$ in a diagram are exactly parallel. Note that "not parallel" is not the same as "lines cross".
On the empirical side, we do not have the $\mu_{jk}$, but can only plot their estimates, the cell means $M_{jk}$. Even if the null hypothesis is true, their connecting lines will almost never be exactly parallel due to measurement error. Conversely, even if the alternative hypothesis is true, they could be almost parallel for the same reason. A measure for the degree to which deviation from parallelity of the $M_{jk}$ indicates interaction is the ANOVA's corresponding F-value.
$\begingroup$
$\endgroup$
Yes, if the true (hypothetical) responses are not parallel there is interaction. Not parallel, however, does not necessarily mean that the segments cross. When you investigate interaction the sampling error may lead to different results in the sample than in the population, so it's useful to calculate confidence intervals or credibility intervals for the extent of the possible interaction. The extent of the interaction depends on the scales of the variables, in special cases (removable interaction) there is a transformation when the effects are additive and there is no interaction.
