2
$\begingroup$

I realize that similar questions have already been asked and answered, but I am in need of a bit more detail and specific advice as I am new to PCA and statistical methods in general. My question is also a bit broader because I will be putting it in context and I need to know if I'm even headed in the right direction.

I have a great deal of data. For each datapoint, there is one continuous response variable that I am interested in examining. Let's call it X.

There are also five or six categorical variables, most of which have between three and ten possible values. One of them, however (let's call it A), has 154 possible values, and to complicate things further, each datapoint can fall in 1-4 of those 154 categories. For the vast majority of them, they just take one of the 154 values, but about 10% of them take two or three values, and maybe 0.5% of them take four values. (I am actually considering including a discrete but quantitative variable that will be equal to the number of values taken by S, as I think it might also be a relevant factor affecting X.)

My ultimate goal here is twofold: to create a predictive model with multiple regression, and to use ANOVA to determine how much each of my variables' variance explains the variance in X.

Someone more familiar with statistics than I suggested that I start with PCA because both multiple regression and ANOVA assume that all factors are independent. I'm pretty sure there are some correlations between a few of my factors (though I have no idea what they are) so I figured PCA would be a good way to begin disentangling.

My questions are:

  1. Can I perform PCA given the categorical nature of my data? If so, what method should I use to "dummy code" the variables? If not, what method would be more effective?

  2. Will including a single discrete, quantitative variable (the number of values taken by A) complicate matters?

  3. Will PCA even do what I want? (namely, disentangling the variables so I can then use multiple regression and ANOVA)

  4. Whatever you recommend, is it possible in R, and if so, how? (I haven't even downloaded R yet but it's been recommended to me and it's free so I'm inclined to give it a swing. I have some programming experience in Python and C++ so in theory I can learn it without too much difficulty.)

Thanks very much in advance.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Neither regression nor ANOVA assume independence of the factors. If the correlation is severe, however, you might consider looking at http://en.wikipedia.org/wiki/Multiple_correspondence_analysis Here is a guide with some of the R packages referenced: http://factominer.free.fr/classical-methods/multiple-correspondence-analysis.html

Improve this answer
$\endgroup$
1
  • $\begingroup$ Part of the trouble is that I don't know how severe the correlation is. I'm not sure how to go about looking for it since I have so many variables and any correlations are likely to be mostly due to chance. As an example (you can read about the context of my project in my comment on Peter Flom's post below), there have historically been fewer seminars on certain days. It's possible that if one of those days appears lower/higher on revenue it could be by chance correlation with another variable like topic. I somehow need to check for--and account for--any such accidental correlations. $\endgroup$
    – Pacific 231
    Commented Jul 18, 2012 at 1:03
1
$\begingroup$

I am a little concerned about a couple points in your post: 1) PCA doesn't have such a thing as a response variable. Sometimes one does PCA in order to reduce the dimension of the set of independent variables in a regression, but it doesn't sound like that's what you are doing.

2) That odd categorical variable is likely going to cause problems. What is it? 154 levels, and some observations are more than one level? It's always easier to answer questions when context is given. As far as I know, PCA has no options for dealing with repeated measures like this.

3) Given that, it would help if you told us what you are trying to do, substantively rather than statistically. You might look at my post How to ask a Statistics Question

Improve this answer
$\endgroup$
5
  • $\begingroup$ Sorry if I wasn't clear enough. I've addressed your points below: 1. The response variable I'm referring to is for the eventual regression/ANOVA. I was hoping to use PCA exactly as you described, just as a way to reduce the dimension of my independent variables set. More importantly, I was hoping to combine them into some new variables to get rid of any correlations. $\endgroup$
    – Pacific 231
    Commented Jul 18, 2012 at 0:57
  • $\begingroup$ 2&3. My data is a set of recorded information on a bunch of seminars that cost money to attend. I'm trying to figure out what factors influence revenue and build a predictive model as well. The factors include things like day of week, topic, etc. The complicated one is the name of the seminar speaker...we draw from a pool of 154 different speakers. Usually just one will host the event, but sometimes we have two, three, or (very rarely) even four. Please let me know if you need any other information. $\endgroup$
    – Pacific 231
    Commented Jul 18, 2012 at 1:00
  • $\begingroup$ First, you probably want regression, not PCA Second, you probably do NOT want to include name of seminar speaker in the list of IVs if your data is structured by attendee rather than by seminar. You might have to account for the non-independence of data. $\endgroup$
    – Peter Flom
    Commented Jul 18, 2012 at 10:27
  • $\begingroup$ Sorry if I didn't explain this well, but my data is structured by seminar, NOT attendee. That is to say, there is one data point per seminar held. $\endgroup$
    – Pacific 231
    Commented Jul 18, 2012 at 13:32
  • $\begingroup$ OK, so some seminars were given by multiple people? That makes sense. But rather than include instructor, I think it might make sense to combine seminar and instructor, when there is more than one instructor. That is, it could be something like History Math Smith Math Johnson Biology $\endgroup$
    – Peter Flom
    Commented Jul 18, 2012 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.