I am performing a survey. There will be one question where I will show an image and ask to identify something in that image. For different participant, the image will be altered in different ways.

As an example, suppose I have 4 methods for image alteration, 40 participants in the survey, so that 10 participants see one type of altered image. The response is binary (wrong/right).

There will be one additional question for each participant: how easy was it to identify the object, with a 7 point likert scale.

I want to understand, which type of alteration was most successful at obscuring the object in the image and which alteration was least successful.

How should I analyze the data?

  • $\begingroup$ You should give responses in table format incorporating your classifacation for data. $\endgroup$ – Subhash C. Davar Mar 22 '17 at 13:26

The key discrete variable that you want to see the difference is the "type of alteration" which contains 4 groups. I am assuming you are not trying to see the difference between the groups of 10 people too and just want to see the difference between 4 groups of alteration. If my assumption is correct, then you might as well show all four types of alteration to all the members (which is 40 in your case or you have the option to decide). By doing so, you are eliminating the personality differences between the groups of 10 people and therefore you can avoid some group of 10 people being biased towards particular type of alteration. (The problem you have mentioned seems to me like more of a within subjects problem than that of a between subjects problem)

Now that we have eliminated personality differences, you might also want to consider if there is a chance that there is carry over effect between the type of alteration. That is, can the same person cause correlation between two types of alteration? This correlation is of two types: symmetric and asymmetric. In symmetric carryover it does not matter if type A is shown first or type B, the carryover is proportional. In such a case within subjects ANOVA is suitable. In asymmetric there will be differences whether type A is shown first or type B is shown. In such a case within subjects design will become error prone and one has to resort to between subjects design.

However, I am assuming there are no such carry over effects in your problem and therefore a ANOVA followed by Tukey HSD test will be the best possible solution. Multiple t-tests between groups can be carried out too but this accumulates type 1 errors and therefore is to be avoided. Whereas ANOVA tells whether there is a difference between types of alterations, Tukey HSD tells which type of alterations differ. After concluding that the types of alterations differ, you can pick the alteration with the least mean (you might want to check the median too if you think the data has outliers)

Summarizing, by the assumptions stated above, I would perform an ANOVA within subjects design, followed by Tukey HSD and pick the one with least means (or median) from the type of alterations. I would also perform individual t-tests between pairs with well defined hypothesis to confirm the results, since Tukey HSD might not give the direction of the difference (although intuitively we know by observing the means)

All the best!

  • $\begingroup$ thanks @bharadwaj. I cannot show all 4 alterations to a single participants. In the 4 alteration types, one is 'no alteration' that is the original image. Only 1/4th of people will see this. and we want to measure, for example, how many people could identify the object in unaltered version and how many could do it in any particular altered version. from this, we can say which is the most effective alteration for hiding the object (compared to the unaltered baseline version) $\endgroup$ – Rakib Mar 21 '17 at 5:09
  • $\begingroup$ I would still design my experiment such that all the four photographs would to be shown to each person and identify how many of them could differentiate between the altered and unaltered version. Maybe the type of photographs in question rule out this possibility because of carryover effects, but then it depends on how you design the photographs. If impossible because the photographs require to be of a certain kind for the experiment, then between subjects design is the way forward. $\endgroup$ – bharadwaj aldur Mar 30 '17 at 6:26
  • $\begingroup$ The subjects within each group need to be sampled carefully so that selection biases for sample together with biases of demographics/psycographics are eliminated between the subjects. A between subjects one way ANOVA followed by a post-hoc like Tukey HSD should suffice :) $\endgroup$ – bharadwaj aldur Mar 30 '17 at 6:26

This is a simple not really statistic answer. You could organize your data in this way. enter image description here

And then from there run some comparative tests, or use graphs to compare the outputs. I think it all matters how the data will be set for analysis, and that will dictate what you can do. I have found this is a good way to set up data for analysis in the past, but there are different ways depending on your goal.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.