I need your help. I'm designing a study for the following problem. There is a feature A (implemented by many software from my specific context) and I'm 100% sure that if I ask the users they will agree that any software (from my specific context) is useless without feature A. Now I want designed software B without feature A, which will be accepted by the users. So basically I want proof that feature A is in general (in my specific context) not necessary.
The study design is the following:
- Survey 1:
- I ask the users if they need feature A, so the answers are "yes/no"
- this question is necessary because there is no data to this question but I'm 100% sure, that most of the users will state "yes"
- Users test my designed software without feature A
- Survey 2:
- I again ask the users if they need feature A, so the answers are again "yes/no" and I expect that the users will change their mind.
Now I want proof that feature A is not necessary. So I have 2 dependent samples (exactly the same users) with dichotomous dependent (feature A necessary: yes/no) and independent (software tested/not tested) variables.
My question is, which statistical test can I use here to proof my hypothesis? As the variables are both nominal only chi-squared homogeneity test comes to my mind. Are there some better tests I can use or can I somehow make a better study design?