Suggestions for identifying key features I have a large set of customer data.  For these customers, I have devised a customer loyalty score which is a measure of the loyalty of the customer.  I want to find the features that are strongly associated/correlated with this score.  Features could be number of purchases at various merchant types.
One obvious answer would to be just to calculate the correlation for each feature with the customer loyalty score and see which have the highest correlations.  Is this preferred way of doing this or are there better techniques?
 A: One way to reformulate your problem is the following: you want to select a small set of features that predict well the loyalty score, using a linear model for example. This problem is called (best) subset selection. 
Suppose that you want to pick k features. The first way to do it is to test all the subsets of k features, by doing linear regression on each subset. But for large dataset, this is way too long.
Another way to do it is in a greedy way. You start by picking the feature that is the most correlated with the score and add it to the (empty) subset. You compute the linear model associated to this subset (in this case, just a coefficient) to predict the loyalty score. Then, you pick the feature which is the most correlated with the residual (the difference between the value predicted by your linear model and the true score) and compute the linear model corresponding to your new subset. You do so, until you have k features in your set.
There are other methods, such as the lasso, to do subset selection. For a more complete introduction to subset selection, you should read the section 3.3 of The Elements of Statistical Learning, which is freely download-able on the authors' site.
A: I understand that the loyalty score is calculated on the strength of some data. If your features include components that are used in calculating the loyalty score they will prove evidently influential.
Multivariate techniques are probably more useful than pairwise correlations:


*

*they can detect weaker features that may be useful in combination with stronger ones

*they can reveal that some features have very similar information content.


The most simple way to start with could be multiple linear regression, although other methods may be better depending on many conditions.
A: Sounds like Business Intelligence work (http://en.wikipedia.org/wiki/Business_intelligence). Could you confirm if it's a customer database or a survey that you ran? Both? Is it from a CRM database? Are customers segmented? Demographically/Physcographically? We need more detail as to what you have. 
If it's a customer database, correlations tell a story about how features load upon your score but not the only story (cor != cause). If you have transactional information you can run survival analysis and calculate life time value (always useful). 
We need to know a lot more about your variables in order to make recommendations of "what to do with it"
A: In addition to the suggestions from the previous answers, I would suggest the catdes function from the FactoMineR package in R. It gives a description of the categories of one factor by qualitative variables and/or by quantitative variables. The output is briefly explained in the manual but I think it would be worth to have a look at the reference mentioned there. The idea is that you get a list of the variables that characterise the most the factor along with a p-value to assess significance.
Note 1 I think that the function is particularly used in a "cluster analysis" context. 
Note 2 It requires to discretise your "customer loyalty score"...
By the way, about three years ago I used that function and I had a question about it. I wrote an email to the author (mentioned in the manual) and he kindly answered me! 
