Survey analysis trouble, looking for references

Question

I'm analyzing a questionnaire in which people were asked about their shopping habits. Respondents were first presented with a set of three qualities a store might have and asked ($Q1_i$, $i=1,2,3$) to rate how important those are when deciding where to shop. For example one factor would be "attractive customer loyalty programs" which the respondent would rate on a Likert scale (1: very unimportant - 5: very important, I treat this as continuous for simplicity currently). Next, the respondents were given a list of five store chains, and for each chain they were asked ($Q2_{ij}$, $i=1,2,3$, $j=A,B,C,D,E$) to rate the chain according to the same factors as before, i.e. whether chain A has 'attractive loyalty programs' etc. (same scale). Finally, ($Q3$) the respondent indicates the chain they spend the most money at on average.

I want to come up with some kind of model: which store has which qualities and what makes people shop there. Unfortunately I don't know much about analyzing surveys at all, so I'm looking for some references or advice. What methods would be standard in such analyses? I'm having trouble finding appropriate learning materials, because I don't know the names of any relevant techniques. I'm not even sure how to title this post.

One thing I came up with would be to consider each factor separately and try to look for statistically significant differences in the average rating ($Q1_i$) between the mutually exclusive groups for which $Q3=A,B,...,E$. So basically running an ANOVA kind of regression for each $i=1,2,3$. Then, for instance, if I found that the average value of $Q1_1$ is significantly larger for the group with $Q3=C$, it would indicate that people who shop at chain C value factor #1 more than the other shoppers. This would lead to a ranking of the store chains for each factor induced by the regression coefficients: the chain with the largest positive deviation could be said to perform best in terms of factor #1.

Does this reasoning make sense at all? It bothers me that I can't think of a way to sensibly include $Q2_{ij}$ in my analysis, since it constitutes most of the information gathered in the survey...

===============================

EDIT: Here is a method I came up with, using Q2. I don't know if it's mathematically justified though:

• Fix factor $i$.
• For every $j$, split the ratings of factor $i$ into two disjoint groups: ratings of company $j$ and ratings of all the other companies.
• Use the t-test to compare the means of the two groups. This gives me a matrix of p-values $M_{ij}$ indicating whether for factor $i$, company $j$ has a significantly different average rating than the remaining companies.

Is this approach correct?

Answer 1

My answer is going to direct you down a less conventional path (though I am drawing from other survey research domains...so there may be some value).  Survey research is used a lot in studying motivation, and one of the early motivation theories posited that two things both had to be present for motivation:  expectancy and value.  Very briefly, to be motivated, you had to value the thing you needed to do, and you had to expect success at the thing you needed to do.  I am certain I could be a good garbage collector, but I don't want to...not motivated to be a garbage collector.  I would value a cure for cancer, but I don't foresee myself being able to come up with a cure...not motivated to study a cure for cancer.  I really want an A in calculus, and I believe if I work hard I can achieve that...I will be motivated to work hard for the A.

I mention this here because it seems that you may have a similar situation.  A preference for a certain store is more likely to be dependent on not just one attribute, but the interaction of two related attributes.  In this case, I value the rewards program, and this store has a good rewards program.

So, the data may be reduced to the appropriate product terms.  And these may be the independent variables for an analysis where the dependent variable is whether store A was chosen as highest choice or not.

I am only guessing that this may relate.  If you would like me to elaborate more, I'd be happy to do so.  Regarding specific references, I would only be drawing on research reports doing a comparable analysis to that suggested in my introductory example.

Answer 2

Reference for doing survey analysis

Reference for hypothesis and null hypothesis

The biggest thing to do now that you have data is to understand what inferences that data supports and what it doesn't. This is an iterative process that begins with a hypothesis.

Here's an example hypothesis:

"I believe that McDonald's is the most favorably viewed chain from the sample."

And then you would test the hypothesis:

"I will create a stacked histogram using the Likert values from Q2 and compare all the restaurants together and see which is the best overall."

You should also write out your assumptions:

"I assume that the respondents have been to these restaurants."

"I assume there are no missing values."

You could then modify the test to account for the assumptions.

"I'll remove from the sample anyone with a $0 answer for Q3 in order to limit my test to only people who have eaten at the restaurants."

"I'll count responses for each restaurant to make sure the samples are of comparative magnitude. I could use the standard deviation of the response counts to set a cutoff +- 1 σ."

Then you perform the test and see that for participants who spent money at these restaurants Wendy's cumulative score was highest and McDonald's was the least. Then you would have a new hypothesis about why that is and you could use Q1 data to test your hypothesis.

This is the basic process being suggested by the users who have commented on your initial question.