Cheng, H., & Liu, S. (2019). Haptic Force Guided Sound Synthesis in Multisensory Virtual Reality (VR) Simulation for Rigid-Fluid Interaction. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 111–119). IEEE. https://doi.org/10.1109/VR.2019.8797906
In this paper, Section 5.3 presents several experiments to assess the effectiveness of their approach. They asks three questions for each scenarios with or without audio feedback:
Can you recognize that you are interacting with the water, despite the rigid lever provides conflicting perceptual cues?
Do you think increasing haptic feedback / haptic and audio feedback can increase reality?
How strong is the sense of reality?
For the first and second question, the participant answer yes or no, so that they record their response as 1 or 0. The last question is the satisfaction score from 1 to 10.
I thought that they should compare the results by using a paired t-test to each question result, because the condition is related to 'audio'.
In their result, they calculate the mean and variance for each question, and compute p-value and t-value for each scenario. (I think that they find the p-value and t-value by comparing two conditions)
(One scenario with audio, Mean (Q1): 0.73, Std (Q1): 0.2095, Mean (Q2): 0.67, Std (Q2): 0.2381, Mean (Q3): 6.33, Std (Q3): 2.5238),
(One scenario without audio, Mean (Q1): 0.33, Std (Q1): 0.2381, Mean (Q2): 0.40, Std (Q2): 0.2571, Mean (Q3): 3.47, Std (Q3): 1.4095)
P-value: 6.431, T-value: < 0.005
They find the statistical significance between two condition by using a one-way Analysis of Variance (ANOVA) test, and they get the reported p-value is less than a significance level of 0.05.
Moreover, they apply the Post-hoc analysis (pairwise comparison between conditions) by using Bonferroni method and Wilcoxon signed-rank test.
How they the p-value and t-value based on this experiment setting? Is their method valid?
