I wanted to determine if a new sample is within reasonable bounds of the mean of an approximately normally distributed sample of points.

I want to use Hotelling's $T^2$ statistic to obtain a $p$-value for the following test: if the mean of a set of data correspond to that of a new point. If not, my new point is at significant distance from the mean of a set of data.

Given that my point is the mean of the data, Hotelling's one-sample T2 statistic can give me a $p$-value to interpret as the probability of obtaining the data if my new sample really is the mean vector. Essentially, this corresponds to finding out whether my point is inside or outside of some level curve of a multivariate (normal?) distribution that corresponds to a $100(1-\alpha)~\%$ interval and fitted to the data set.

I compute the $T^2$ statistic as $$ t^2 = n(x_{new} - \mu)^t\left(\frac{1}{n-1}X^tX\right)^{-1}(x_{new} - \mu), $$ where the data matrix, $X\sim\mathcal{N}(\mu, \Sigma)$ is $n\times 2$, where $\Sigma$ is a $2\times 2$ diagonal matrix, and then use the $F$ distribution with $2$ and $n-2$ degrees of freedom to obtain a $p$-value. This since $$ \frac{n-2}{2(n-1)}t^2 \sim F_{2, n-1}. $$

Now, my problem is that the $p$-values I obtain are waaay too small. They can be on the order of $10^{-12}$, for points that should have a $p$-value about roughly 0.5.

In order to plot a $95~\%$ confidence interval around the mean, I used the equation for an ellipse $$ \left(\frac{x}{a}\right)^2 + \left(\frac{y}{b}\right)^2 = d^2, $$ where $a$ is the length of the major axis, $b$ is the length of the minor axis and where $d^2=F_{\alpha,2,n-2}$ is the critical value of an $F$ distribution with $2$ and $n-2$ degrees of freedom on the $\alpha$ level.

The figure illustrates 100 normal random samples with $\sigma_1=2$ and $\sigma_2=1$. The ellipses correspond to $50~\%$ and $95~\%$ regions, respectively. enter image description here

It clearly works as expected. However, I realised that the $T^2$ statistic is a generalisation of the equation for an ellipse, which leads me to the conclusion that my value of the $T^2$ statistic is a factor $n$ too large, since we should have $$ x_{new}^t\left(\frac{1}{n-1}X^tX\right)^{-1}x_{new} = \left(\frac{x_{new,1}}{\sigma_1}\right)^2 + \left(\frac{x_{new,2}}{\sigma_2}\right)^2, $$ when the data is centred and the covariance matrix is diagonal.

If I remove the factor $n$, the point $x_{new}=[1, 1]$ obtains a $p$-value of $0.5032$ instead of $0$ (!).

Another observation I made is that when drawing the ellipse, I must use $d$ instead of $d^2$ times the standard deviation of the coordinate. This makes sense, since what I am really computing is a Mahalanobis distance, but it does not correspond to the theory of the confidence interval using the $T^2$ statistic.

So, my question is: What have I done wrong? Why is my computed value wrong by a factor $n$? Should I use the square root of $F$?

  • $\begingroup$ I stopped reading early on because "$x_\text{new}=\mu$" is not a null hypothesis at all and therefore cannot possibly have a meaningful p-value associated with it. It is a prediction, because $x_\text{new}$ is a random variable. Upon skimming the rest of the post, I suspect everything else that follows--being based on this misconception--may be misdirected. Consider instead searching our site for information about multivariate outliers and the Mahalanobis distance. You might also want to review material on null hypotheses, and p-values, and prediction limits. $\endgroup$ – whuber Jun 11 '15 at 13:33
  • $\begingroup$ Thank you, yes that was wrong. I have tried to clarify it. Please bare with me and read the rest; hopefully it will be clear by the end of the question. $\endgroup$ – Tommy L Jun 11 '15 at 13:48
  • $\begingroup$ Although you have rephrased the question--and I thank you for that--you have not changed the misconceptions about p-values on which it is founded. I would like to suggest that you forget momentarily about p-values and focus on the essence of the problem, which you have made clear: is a new point consistent with previous ones? That can be framed as a prediction limit problem, as an outlier test, and--if you make a very strong assumption about the distribution that underlies the new point--as a hypothesis test. That would open up the possibility of creatively different approaches. $\endgroup$ – whuber Jun 11 '15 at 13:55
  • $\begingroup$ Could you please explain the between a regular t-test where the null is if the mean is e.g. equal to some constant mu, or in a $T^2$ test, where it is whether the mean of the data is equal to some constant vector mu? I am not a statistician, but I thought I had this covered at least... I thought it would be to just integrate the multivariate normal probability density function outside of the Mahalanobis distance d from the mean of the data, and that would be the p-value, but then I found out about Hotelling's $T^2$ and its connection to the F and $\chi^2$ distributions, which seemed easier... $\endgroup$ – Tommy L Jun 11 '15 at 14:01
  • $\begingroup$ I might have understood what you meant now. I have made another update... $\endgroup$ – Tommy L Jun 11 '15 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.