I have a set (cluster) of vectors in dimension d. From this I have calculated the sample mean and covariance matrix ( I make the assumption that they are from a multivariate Gaussian).

My question is, given a new vector (in dimension d) I am trying to decide if it belongs to this cluster by checking if the distance from the mean is less than 2 standard deviations.

In the one dimensional case I would simply check if x-x_bar > 2*sigma.

How does this extend to the multivariate case?



First of all, in the univariate case (when $d=1$, e.g. the one you already know the decision rule for), assuming you have a vector of $n$ univariate measurements $x$ (so that $x$ is a $n\times 1$ matrix where each entry $x_i$ is a scalar), the decision rule you describe is really:

$$\left(\frac{n(n-1)}{(n-1)(n+1)}\frac{\left(x_i-\hat{\mu}_x\right)^2}{\hat{\sigma}^2_x}\right) > F_{0.95}(1, n-1)$$

where $F_{0.95}$ is the 95 percentile of a Fisher distribution (you consider that $x_i$ is too far from $\hat{\mu}_x$ in the metric $\hat{\sigma}$ to belong to the cluster with mean $\hat{\mu}_x$ and scale $\hat{\sigma}$). This is the correct version of your rule of thumb when $p=1$ (I denote $p$ what you write $d$, sorry for the confusion but if I change my notation now, my answers to your comments below will become meaningless)

In the multivariate case (where $p>1$, e.g. the one you are really interested in), this becomes: assuming $X$ is of dimensions $n\times p$ (so that each row $X_i$ of $X$ is a $p$-vector) and $\sigma_X^{-1}$ (the inverse of the variance covariance matrix of the $X$) exists:

$$\left(\frac{n(n-p)}{p(n-1)(n+1)}\left(X_i-\hat{\mu}_X\right)'\hat{\sigma}_X^{-1}\left(X_i-\hat{\mu}_X\right)\right) > F_{0.95}(p, n-p)$$

denoting $\mu_X$ the $p$-vector of means of $X$. $\left(X_i-\hat{\mu}_X\right)'\hat{\sigma}_X^{-1}\left(X_i-\hat{\mu}_X\right)$ is the vector of Mahalanobis distances of $X_i$ w.r.t. to $(\hat{\mu}_X,\hat{\sigma}_X)$

  • $\begingroup$ I thought in the univariate case the vectors are of length 1? $\endgroup$ – Aly Feb 11 '13 at 17:17
  • $\begingroup$ it's a notation convention thing. I've made the dimensions explicit in all cases. $\endgroup$ – user603 Feb 11 '13 at 17:27
  • $\begingroup$ But can you use this formula for a multi-dimension vector? for my purpose I have a a cluster of n-dimension vectors and I wish to compute a mean and std deviation so that given another n-dimension vector I can calculate the likelihood that it belongs to this cluster. Should I be using a multivariate gaussian or have I misunderstood something and can just use the univariate? $\endgroup$ – Aly Feb 11 '13 at 17:30
  • $\begingroup$ If your $x_i$ is a scalar, use the first inequality. If your $X_i$ is a vector (of p measurments) use the second inequality. Can you explain where my formulation is confusing? I can edit the answer $\endgroup$ – user603 Feb 11 '13 at 17:37
  • 1
    $\begingroup$ @Aly: the $n$ refers to the sample size used to estimate the parameters ('x_bar' and 'sigma').... $\endgroup$ – user603 Feb 12 '13 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.