# Confusion related to multicollinearity, FA and regression of heterogeneous data

I am currently working with a data set that contains about 26 IVs of almost all sorts of scale of measurement (binary, nominal, ordinal and interval scale variables). There are strong reasons to suspect that some variables are probably highly correlated, while some may not be related to any other IVs to a great extent.

I came across with great suggestions to resolve this problem in this site (which was an useful advice to use optimally scaled variables in the FA procedure and to use derived factor score as the IVs). But due to my inexperience in this field I am in need of some expert advice on the following issues:

How should I check if Multicollinearity really exists?

I am not sure how to check Multicollinearity with such a heterogeneous data. I may calculate the Heterogeneous Correlation Matrix (or Spearman's Rank Correlation) by somehow forcing me to consider the nominal variables as ordinal but even if I do it what should be the value of the correlation coefficient at which Multicollinearity can be ignored? I am also not sure if it is going to give any insight at all, as I am missing something like a VIF measure!

Should I take only those variables for a FA which are highly correlated?

Say, if I can find two sets of variables (one set containing 8 IVs and another containing 4 IVs) quite highly correlated to each other within each set, then should I use only those 12 variables for FA and derive FA scores for those two factors to use them as IVs? Clearly my intention is to use the other 14 variables separately as IVs along with the two derived scores. I am confused if I should actually use not the 12, but all 26 variables in the FA in this scenario. Remember in that case my FA scores are weighted by the other 14 unrelated variables too!

Is there any problem to categorize a proportion type DV for an ordinal logistic regression?

I've actually found people using logistic regression instead. But I want to mention here that unfortunately I don't know the number of cases (or trials) out of which each proportion was calculated. So I cannot use the number of trials as the weights in the logistic regression. In that case a logistic regression may not be accurate enough. So, as I only know the proportions, won't it be good to categorize the proportions by median split or by quartile split? So that I can use it as a DV in a logistic or in an ordinal logistic regression?

Regards.

-
Clarifications-1 From the CATPCA procedure, it seems like only two factors can be found that loads highly onto more than one variables (one factor loads on 8 variables and the other loads on 4 variables). But during the calculation of scores of these two factors, variables which do not load on more than one variable, also have a contribution. Besides the 12 variables that have high loads on the two clearer factors also have a contribution while calculating factor scores of the factors that has a single variable load. – Blain Waan Jan 17 '13 at 5:58
Clarification-2 In the 3rd query I want to know if cut points will affect the regression results. I explained why I am thinking of categorizing (I do not know the number of trials). The DV ranges from 0 to 0.75. The median is 0.10. Quartiles are: 0.05 (25th percentile), 0.10 (50th percentile), 0.17 (75th percentile). – Blain Waan Jan 17 '13 at 6:05
@mbq do I need to provide anything else? It seems like none is trying to improve the answer! – Blain Waan Jan 19 '13 at 22:55

I hoped that someone would come up with a brilliant answer, but in the mean time, here are my thoughts. True multicollinearity is not something you test for, in my experience. It just happens. You try to fit a model and the computer complains, because it's trying to invert a singular matrix (or nearly singular) and it has trouble converging. When the variables are nearly collinear, you start to get weird behaviour. The parameter estimates become unstable, so if you make small changes to the inputs, you get big changes in the parameter values.

But perhaps what you are interested in doing is looking to reduce the dimension of the problem. You can try several things. Look for a best subset of regressions, if you want the variables as they are. Do a principle components analysis and see if most of the variation is explained by a small combination of same. I wouldn't do FA unless I believed that the observations expressed genuine latent factors.

A cluster analysis is another way to look at groupings in the variables.

You didn't say how big your sample is, but if it's close to the number of variables, you will get collinearity for sure, or something close to it. However, that would not mean that the variables were intrinsically related. i.e. in a bigger sample, those 26 variables might not be collinear.

You have not said anything about a model, hypothesis, or anything else, so it's hard to advise beyond what I have said.

-
Thanks for your response. My sample size is 169. A categorical principal component analysis shows that the total variation can be explained with less than the actual dimension. The reason I used the optimally scaled variables from CATPCA in a factor analysis procedure is only to have the benefit of rotation. It seems like only two factors can be found that loads highly onto more than one variables (one factor loads on 8 variables and the other loads on 4 variables). That's why I asked the 2nd question. I hope you will suggest me about the rest of the queries too. Regards. – Blain Waan Jan 16 '13 at 12:47
I'm not sure what you are doing in query 2 about only using correlated FA's, so I did not comment. Regarding logistic regression, that assumes you have a dichotomized response variable (or variable that you can dichotomize). Then you have all the same issues with collinearity and model selection that you would have with the usual multiple regression. – Placidia Jan 16 '13 at 15:49