What test should I use for to analyze my data I am performing a survey. There will be one question where I will show an image and ask to identify something in that image. For different participant, the image will be altered in different ways.
As an example, suppose I have 4 methods for image alteration, 40 participants in the survey, so that 10 participants see one type of altered image. The response is binary (wrong/right).
There will be one additional question for each participant: how easy was it to identify the object, with a 7 point likert scale.
I want to understand, which type of alteration was most successful at obscuring the object in the image and which alteration was least successful. 
How should I analyze the data?
 A: The key discrete variable that you want to see the difference is the "type of alteration" which contains 4 groups. I am assuming you are not trying to see the difference between the groups of 10 people too and just want to see the difference between 4 groups of alteration. If my assumption is correct, then you might as well show all four types of alteration to all the members (which is 40 in your case or you have the option to decide). By doing so, you are eliminating the personality differences between the groups of 10 people and therefore you can avoid some group of 10 people being biased towards particular type of alteration. (The problem you have mentioned seems to me like more of a within subjects problem than that of a between subjects problem)
Now that we have eliminated personality differences, you might also want to consider if there is a chance that there is carry over effect between the type of alteration. That is, can the same person cause correlation between two types of alteration? This correlation is of two types: symmetric and asymmetric. In symmetric carryover it does not matter if type A is shown first or type B, the carryover is proportional. In such a case within subjects ANOVA is suitable. In asymmetric there will be differences whether type A is shown first or type B is shown. In such a case within subjects design will become error prone and one has to resort to between subjects design.
However, I am assuming there are no such carry over effects in your problem and therefore a ANOVA followed by Tukey HSD test will be the best possible solution. Multiple t-tests between groups can be carried out too but this accumulates type 1 errors and therefore is to be avoided. Whereas ANOVA tells whether there is a difference between types of alterations, Tukey HSD tells which type of alterations differ. After concluding that the types of alterations differ, you can pick the alteration with the least mean (you might want to check the median too if you think the data has outliers)
Summarizing, by the assumptions stated above, I would perform an ANOVA within subjects design, followed by Tukey HSD and pick the one with least means (or median) from the type of alterations. I would also perform individual t-tests between pairs with well defined hypothesis to confirm the results, since Tukey HSD might not give the direction of the difference (although intuitively we know by observing the means)
All the best!
A: This is a simple not really statistic answer.
You could organize your data in this way.

And then from there run some comparative tests, or use graphs to compare the outputs.  I think it all matters how the data will be set for analysis, and that will dictate what you can do.  I have found this is a good way to set up data for analysis in the past, but there are different ways depending on your goal.
