I am not so familiar with statistics, since I am a computer science student. I hope it is alright to ask this question here.
I am currently working towards the end of a thesis. I compared the User Experience of Android Apps and Progressive Web Apps (PWA- basically Websites which act like normal apps) and the research question was if PWA can keep up with android apps in terms of UX.
For measuring UX I conducted a user study and used an existing questionnaire (UEQ), which calculates different scales of UX.
To find out if a significant difference exists between those apps I performed t-tests for each scale (calculated value).
Every calculated p-value was way above the alpha-level of 0.05, so I reject the null hypothesis for every scale. At the beginning I thought everything was fine -> no statistical difference found -> PWA can keep up.
But the more I read about this topic I found out a high p-value does not indicate that the groups are equal or that there is no effect. Other sources say "no statistical significance" just means that the Null hypothesis cannot be rejected and "anything is possible".
Question: Are the results of the questionnaire of any use to answer my research question or do I have to say "anything is possible"?
In addition I conducted a qualitative study, which concluded that both types of apps were able to offer the same UX, but now I am doubting if I can put these result in relation to the results of the quantitative questionnaire.
Originally I thought you could use the null hypothesis as a result but I guess I was wrong. A number of research papers, which had a similar topic, just said "no statistical significance has been found, therefore we conclude that both types of Apps were able to ........", which seems to be logically but not mathematically correct.