1
$\begingroup$

I currently have a data frame with 98 observations and 107 variables. All of the variables are numeric, but one variable is binary (yes or no). My goal is to determine which correlation and/or variable give the greatest segregation between the yes and no samples. I have been using the pairs () function to do this, but I can only do a few variables at a time. Is there a way to determine which correlation gives the greatest discernment between yes and no?

To Clarify - My table is 98 observations and 107 variables, but doing a correlation matrix with the pairs function is not able to fit all of the variables.

I have used this function:

pairs(x[70:80], ch=21, bg=c("red","green")[unclass(x$outcome)])
$\endgroup$
3
  • $\begingroup$ don't you want ...to determine which VARIABLE give the greatest segregation between the yes and no samples? (instead of correlation) $\endgroup$ Oct 16, 2013 at 18:34
  • $\begingroup$ Data frame with 98 observations and 107 variables sounds like simple table (no correlation matrix). Please clarify! $\endgroup$ Oct 16, 2013 at 18:37
  • $\begingroup$ One idea is stats.stackexchange.com/questions/190156/… but in your case use regularization $\endgroup$ Nov 17, 2022 at 17:26

3 Answers 3

6
$\begingroup$

When you have multiple variable and you are looking for variable(s) which is the best for discriminating between groups ("yes" and "no" samples in this case) a tool for this is MANOVA.

# Suppose we have a data.frame with 7 variables and one group:
my.data<-data.frame(v1=rnorm(100),v2=rnorm(100),v3=rnorm(100),
v4=rnorm(100),v5=rnorm(100),v6=rnorm(100), v7=c(rnorm(50),
rnorm(50)+20),response=rep(c("yes","no"), each=50))

# run MANOVA
my.mnv<-manova(cbind(v1,v2,v3,v4,v5,v6,v7) ~ response, data=my.data)

# and look on p-values (if p-value < 0.05 then it is able to 
# significantly discriminate between "yes" and "no")
summary.aov(my.mnv)

# plot
pairs(my.data[c("v1","v2","v3","v4","v5","v6","v7")], pch=22,
bg=c("red", "yellow")[unclass(my.data$response)])

It's not good to make conclusions about statistical significance based on looking on the plot (although it is necessary to look on it). In you case of 107 variables the pairs() plot will be very chaotic.

$\endgroup$
5
  • $\begingroup$ Thanks Ladislav, I ran the approach and I also confirmed it with the plots and it worked well. Thanks again $\endgroup$
    – GK89
    Oct 17, 2013 at 16:29
  • 2
    $\begingroup$ I do not agree AT ALL. MANOVA seems appropriate only for the configuration of the problem, but: (1) with 107 variables (and just 98 observations), statistical power will be non-existent, (2) are you prone to assume multivariate normality in a 107-dimensional space?, (3) Strong multicollinearity is expected among 107 variables (even if just random ones), (4) you will have no further knowledge of the variables "causing" group discrimination. Considering, focus in p-values is simply not-right. $\endgroup$
    – FairMiles
    Oct 28, 2013 at 22:37
  • 1
    $\begingroup$ There is no automagic shortcut to dataset knowledge by exploration in cases like this one. And, then, variable selection. With a similar "shape" and assumptions you may start by analyzing paired correlations for redundant variables and single t-tests for uninformative variables. Then try to detect important variables by exploring the main (or first) linear combinations of a (mathematically equivalent to MANOVA, so same assumptions apply) discriminant analysis (DCA) $\endgroup$
    – FairMiles
    Oct 28, 2013 at 22:42
  • 1
    $\begingroup$ @FairMiles thank you for your comment. I agree that focusing ONLY on p-values is not good and MANOVA is probably appropriate in this particular case. Could you please write an answer in which you will use DCA (in R)? Such answer/example will be very welcome. $\endgroup$ Oct 29, 2013 at 7:18
  • 1
    $\begingroup$ you may earn bounty also... $\endgroup$ Oct 29, 2013 at 7:19
4
+100
$\begingroup$

An alternate method of finding variable importance is using random forests.

A package called varSelRF was built specifically for this purpose. This method isn't designed to be right all the time, but is a rather quick way of dealing with large dimensions to get a semblance of which variables could at the first level affect the response variable.

Combine this with an MANOVA and you stand a decent chance of finding your key variables.

$\endgroup$
3
  • $\begingroup$ Thanks for answer @Arun Jose. Can you please add an example written in R? Just small, demonstrative... $\endgroup$ Oct 31, 2013 at 10:49
  • $\begingroup$ Good point! And thank you for the package recommendation $\endgroup$
    – GK89
    Oct 31, 2013 at 20:22
  • 2
    $\begingroup$ Ladislov, I resist providing a code snippet as best practice would be to simply try out the examples in the package documentation. It would help you immensely as you start to understand the limitations and strengths of the algorithm. I'd suggest simulating a dataset to begin playing around with it, and then once you know how it works, start mining the data you really need. $\endgroup$
    – Arun Jose
    Nov 3, 2013 at 17:13
3
$\begingroup$

I would advise against using statistical methods for determining the "best" variable. You have only 98 observations (how are "yes" and "no" answers distributed?) and more variables than cases. This is a recipe for disaster in the sense that any attempt to build a model with all variables is prone to overfit the data. You will find packages that try to do the trick and some careful cross-validation might help you to avoid some obvious pitfalls, but do not assume that you will learn much on a conceptual level. My suggestion would therefore be to eliminate variables that are weak on theoretical grounds before moving to the analysis step or to collect more cases if that is possible. And: test some simple models (equal weighting of variables) as competitors to get some benchmarks.

There is less literature on the soft and fuzzy process of conceptual variable selection than on algorithmic ways to "solve" this problem, but this is not necessarily due to the superiority of the latter. Some pointers in the literature could be:

Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American psychologist, 34(7), 571.

Freedman, D. A. (1991). Statistical models and shoe leather. Sociological methodology, 21, 291-313.

Freedman, D. (1999). From association to causation: some remarks on the history of statistics. Statistical Science, 14(3), 243-258.

$\endgroup$
2
  • $\begingroup$ Yes, thought-out design of an experiment is essential to any scientific research. @jank could you please add some links which you think could help with thinking before data collection procedure? $\endgroup$ Nov 1, 2013 at 11:29
  • $\begingroup$ I edited the text so that it now includes some references. $\endgroup$
    – jank
    Nov 1, 2013 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.