2
$\begingroup$

I am doing multivariate analysis of variance for a dataset composed of m observation in n dimensions (n>>m, that is n>10000 and m=20). the problem is that it looks like that I have some variables are linearly dependent as I got the following error message:

The within-group sum of squares and cross products matrix is singular.

So, what is the best solution for such problem? (applying PCA and SVD might not be suitable as the covariance matrix will be 10000x10000 and will be difficult to process it)?

$\endgroup$
3
  • $\begingroup$ Nah, I'm just working on a 38 x 500000 (500K) data set, PCA and even more complex methods work beautifully. What do you want to test with your ANOVA? Are there groups of samples? $\endgroup$
    – January
    Commented Jul 5, 2013 at 13:06
  • $\begingroup$ these 20 observations belong to two different groups (10 samples per group) and I want to test if the samples of the first group are more dispersed (differ between each other) than the other group. (I tried Matlab PCA on my workstation but it gives out of memory. Do you use some other packages)? $\endgroup$
    – Abbas
    Commented Jul 5, 2013 at 13:20
  • 1
    $\begingroup$ I'm using the standard prcomp function from R. $\endgroup$
    – January
    Commented Jul 5, 2013 at 13:53

2 Answers 2

1
$\begingroup$

The best solution depends on the exact nature of the problem.

One literature you might want to look into is the one around "age-period-cohort models". We might expect that someone's age, the era in which people grow up (the cohort), and the period in which the question was asked all have their own influence on a person behavior, but they are linearly dependent: year of birth (cohort) = period - age. There is a whole set of models that put different constraints on the different parameters in order to try to identify the different effects. These models are often refered to as "age-period-cohort models".

$\endgroup$
2
  • $\begingroup$ As I did not know about it before, I am not sure if "age-period-cohort models" is applicable for the nature of my problem, which is: my dataset represents 20 observations (samples) belong to two different groups (10 samples per group) and the variables represent the expression of genes in each sample (gene expression matrix). I do not know if the concept of "age-period-cohort models" is still applicable for such problem? $\endgroup$
    – Abbas
    Commented Jul 5, 2013 at 13:33
  • $\begingroup$ might be, might not. The idea behind these models is that the variables that are linearly dependent are all substantively different and interesting. If that applies to your case, then I would say yes. If not then I would consider dropping variables that you don't find interesting. Since the variables are linearly dependent you don't loose any information when you do. $\endgroup$ Commented Jul 5, 2013 at 13:37
0
$\begingroup$

Why do you think MANOVA will answer your question? It will give you the (linear) combination of the m variables that maximize the difference between the centroids (k = 2, in your case) assuming that dispersion is the same (and that everything is multi-normal...).

So that seems almost the opposite of your interest of testing for differences in dispersion (as you commented below your question). By the way, as far as I know you need m (per group) > n to get the eigenvalues and eigenvectors. Without m >> n you won't have enough power for significance tests.

If what you want to resume what all those variables have to say about describing your observations, then PCA may be useful as an indirect method (i.e., first resume those variables in independent components that maximize the explanation of samples heterogeneity, then compare groups based on those components) and linear correlation among variables is not a problem there (is what the components are all about). The particular solution (e.g., important variables on the first component) will be very unstable, anyway, because n << m.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.