Analyse within-subject data as between-subject data Presence (or better sense of presence, i.e. the extent to which one feels that a mediated experience is not actually mediated) is a notoriously difficult variable to pin down and the question of how to measure it is hotly debated in its respective community. One particular debate concerns the the validity of standard self-report questionnaires used to measure a subject's sense of presence. 
In a between-subjects experiment, people performed a search task in either a real or a virtual office. They then rated their sense of presence on a set of standard questionnaires. Participants in the virtual condition self-reported a similar sense of presence than participants in the real world condition. 
While the authors only conclude that presence questionnaires should pass a "reality test" in order to be useful, I personally think that the results might have been different in a within-subjects experiment. Presence is such a subjective and abstract concept to the layman that I think it unlikely that subject A's feeling of presence under condition X can be meaningfully compared to subject B's feeling of presence under condition Y, but that condition X gives a subject a frame of reference on which to evaluate condition Y, if the subject is exposed to both (Note, IMO this would entail that presence could only be regarded a measure for relative differences between conditions rather than an absolute scale). This is somewhat substantiated by the fact that answers on presence questionnaires have a notoriously high variance.
I have already designed an experiment (which has four different conditions) as a within-subjects experiment using 48 subjects and balancing the order of conditions across subjects.
Then I got to wonder whether I could test my assumption with my data. So I am thinking about analysing my data as within-subjects data (i.e., using a repeated-measures ANOVA) but also taking the very first condition each subject is exposed to and simply treat that subset of the data as if it came from a between-subjects experiment.
However, I am wondering whether this is a valid statistical approach and what statistical test would be appropriate, rather than simply stating, for example, that in the within analysis condition X was different from condition Y while in the between it was not.
For example, would it be valid to treat mean for condition X using within data and mean for condition X using between data (for all four conditions) as factors for an ANOVA to see if the mean for a condition significantly differs for a within and a between subjects experiment?
NB: I could potentially add more subjects which only get exposed to one condition in order to equalise the number of data points available for the within and between subjects analysis but I would prefer to do the analysis on the original subjects only, as (a) the experiment is quite lengthy and cumbersome, and (b) I was actually hoping that keeping the same pool of subjects for both analyses would give me a stronger indication of whether my hypothesis holds, as it eliminates any variability due to some subjects appearing in one set of data but not the other.
 A: It's not a very sound argument you have for your hypothesis... from what you said.  Having a reference point of being in other conditions might make subjects less variable, and the effects observed across conditions within subjects should be much less variable.  You should get more power in the within design.  But I don't see any argument here at all for why they would switch direction of their effect.  You don't have any foundation for that unless you're arguing that the between design previously run is just a spurious result.  Check the power of the study and see if that's likely.
Let's say you have two condition A, and B and you do them repeated measures with 4 subjects like this.
trt1 trt2 order
A    B    1
A    B    1
B    A    2
B    A    2

Now, I'm not sure what you predict but it seems to me that you're saying B>A in the literature but you think A>B with repeated measures.  That means that you should replicate the literature in column 1, and then also A>B in column 2.  You'd need that in column 2 to get your repeated measures effect (assuming the literature replicates in column 1).  But then you really are going to need something more in your explanation than you have now because you'll have replicated what, as near as I can tell, you're just calling a spurious finding because people don't understand presence until they've experienced multiple conditions.
A: The analysis you want to pursue is not uncommon (unfortunately I don't have a reference right now), but you have to split it.
You have to run 2 ANOVAs and hope that the analyses agree. 


*

*The overall ANOVA uses the full within subject design.

*The between-subjects ANOVA just uses the data from the first session and compares those.


If the seond analysis is not significant but shows the same pattern, this may be, as there is less power.
