2
$\begingroup$

In a comparative study on human motion and perception in both reality and virtual reality, I am examining differences and relations between specific entities.

In two separate experiments, TTG Estimation* and Distance Estimation* data were collected from the same set of test persons who had following tasks:

  • *TTG Estimation task: Estimation for a Time-To-Go in [seconds] - test subjects are asked to signal the point of time (in Reality and Virtual Reality) at which they feel there's just not enough time for securely crossing the street until the arrival of a vehicle (at 3 different speeds).
  • *Distance Estimation task: Test subjects are asked to estimate the distance in [m] between them and a pylon (in Reality and Virtual Reality)

Following table shall give a visualisation of the experiment design with the corresponding variables to be examined:

+-----------------+-----------------------------+---------------------------------+
|                 | TTG Estimation Experiment   | Distance Estimation Experiment  |
+                 +-----------------------------+---------------------------------+
|                 | 30 km/h | 35 km/h | 40 km/h | 8m | 9m | 10m | 11m | 12m | 13m |
+-----------------+---------+---------+---------+----+----+-----+-----+-----+-----+
| Reality         |         |         |         |    |    |     |     |     |     |
+-----------------+---------+---------+---------+----+----+-----+-----+-----+-----+
| Virtual Reality |         |         |         |    |    |     |     |     |     |
+-----------------+---------+---------+---------+----+----+-----+-----+-----+-----+
| Test Variables  |     abs(TTG_R - TTG_VR)     |      abs(Dist_R - Dist_VR)      |
+-----------------+-----------------------------+---------------------------------+

So in general I'm interested in the answer to the following question:
Is there a relation between the estimation ability/quality for TTG and the estimation ability/quality for Distance? Or in other words ...
Do test persons with a good estimation for the TTG also give a good estimation for the Distance?

So my questions are:
a) What hypothesis approach is more appropriate? A comparison between two test variables OR a correlation analysis?
I am confused by the fact that the test variables |TTG_R - TTG_VR| and |Dist_R - Dist_VR| have different levels (TTG: 3 different vehicle speeds; Distance: 6 different distances) and that the test variables are measured in different units (TTG: s; Distance: m).

b) What exact method/test can I use to adress my point of interest? Does difference hypothesis make sense? (e.g. ANOVA for multiple factors?)
I have thought of normalising the test variables via zscore and check for bivariate correlation but then immediately I realise there are different test levels for each variable TTG: 3 different vehicle speeds; Distance: 6 different distances).
And regarding a difference hypothesis I am not sure if test levels can be used as factors?

Available software for evaluation:

  • Matlab
  • SPSS
$\endgroup$
6
  • $\begingroup$ I can't imagine approaching a (psychometric) problem like this without employing latent factor analysis. Hierarchical Bayesian models are ideal for stuff like that. John Kruschke's 'puppy' book (now in 2nd edition) might be a great place to start. (The 2 software packages you listed probably are not!) $\endgroup$ Commented Apr 14, 2017 at 11:55
  • $\begingroup$ If you had to think of one of your measurements as the "outcome" and the other as a "predictor", which would be which? Would you be more interested in assessing whether TTG performance is predictive of Distance performance, or more interested in assessing whether Distance performance is predictive of TTG performance? $\endgroup$
    – rcorty
    Commented Apr 14, 2017 at 16:15
  • $\begingroup$ Is the distance from the point at which the vehicle starts moving to the point where the participant crosses the street known? What is the TTG? Is it the participant's estimate of the time it will take them to cross the street? Or is it the time that it takes them to cross the street? (In the latter case I would be particularly interested what happens to participants in the reality condition if they misjudge the TTG). $\endgroup$
    – matus
    Commented Apr 14, 2017 at 16:48
  • $\begingroup$ @matus see edited question for the TTG estimation in reality; Yes the distance from the point at which the vehicle starts moving to the point where the participant crosses the street is known. $\endgroup$ Commented Apr 14, 2017 at 19:54
  • $\begingroup$ @DavidC.Norris Thanks for the hints!! SPSS does indeed have a function for factor analysis, is it not good enough? Could you elaborate a bit? thanks! $\endgroup$ Commented Apr 14, 2017 at 19:57

2 Answers 2

2
$\begingroup$

So, the STAT 101 approach this problem would be a Pearson's r correlation test between error-in-distance and error-in-time. And as long as you have at least a few tens of subjects and the errors in each variable look more-or-less bell-shaped, this is probably still the first thing I would do with this problem.

Something like a paired t-test would definitely be wrong. Since the two measurements have different units, there is no way that the same-variance assumption could be satisfied. In practice this would show up as your result being unit-dependent.

ANOVAs are about comparing treatments where the independent variable levels are non-orderable and discrete and the dependent variable levels are orderable and continuous. So an ANOVA treatment is going to treat your distance and time errors in a fundamentally different way, even though they are fundamentally comparable. Also, since ANOVAs don't know anything about the ordering of the independent variable, they are not going to be as powerful.

Really, you have screwed yourself a bit by ordinal-zing variables that are fundamentally continuous, that is by binning your measurements. Making that choice is what led you consider not-really-appropriate ANOVAs, and technically it rules out continuous-variable tests like the r-test I suggested. But as long as the discrete-ness isn't too bad, it's pretty common practice to use continuous variable techniques on ordinal variables. I don't suppose you can go back to the original data and un-bin?

More complex stuff like latent factor analysis (which is not the same thing as "factor analysis" in SPSS) is basically fitting a parametric model, which can take into account other things you know about your estimators (sex, age, etc.) It's a fine next step, but there is nothing wrong with answering your first question with a simple correlation test.

$\endgroup$
2
  • $\begingroup$ Unfortunately, I only got these data from someone else, so I can't change anything in terms of experiment design. Could you please shortly explain what you mean by binning? $\endgroup$ Commented Apr 19, 2017 at 8:57
  • $\begingroup$ By binning, I mean putting the errors into bins of "off by around 10 m/s", "off by around 15 m/s", etc, instead of just recording "off by 12.7 m/s", etc. for each estimate. $\endgroup$ Commented Apr 19, 2017 at 21:58
1
$\begingroup$

I don't think that linear methods or measures applied to the observed variables at the raw scale will be useful, since time, distance and speed are all positive quantities and a basic result from newtonian mechanics tells us that their relationship is multiplicative, not additive. In particular, we know that

$$t=\frac{d}{v}$$

With the help of this equation we can translate the time judgments in TTG experiment to distance. In particular, $d_\mathrm{ttg}^\mathrm{estim}=t_\mathrm{ttg}^\mathrm{estim} \cdot v_\mathrm{ttg}^\mathrm{estim}$ where $v_\mathrm{ttg}^\mathrm{estim}$ is a speed estimate and is not observed. The true distance $d_\mathrm{ttg}^\mathrm{true}$ is the distance of the vehicle to the crossing at the time when the TTG jugment is made which I assume is known. In this way we obtain distance measures in the TTG experiment analogous to the Distance estimation experiment. $v_\mathrm{ttg}^\mathrm{estim}$ is not known. Perhaps one could assume that observers are accurate in their estimate $v_\mathrm{ttg}^\mathrm{estim}=v_\mathrm{ttg}^\mathrm{true}$. Otherwise one would treat $v_\mathrm{ttg}^\mathrm{estim}$ as unknown parameter.

At this point one may utilize any of the models that are suitable for strictly positive variables. A simple approach would be to assume $v_\mathrm{ttg}^\mathrm{estim}=v_\mathrm{ttg}^\mathrm{true}$, to compute $d_\mathrm{ttg}^\mathrm{estim}$, to log-transform the distances $d_\mathrm{ttg}^\mathrm{estim}$, $d_\mathrm{ttg}^\mathrm{true}$,$d_\mathrm{de}^\mathrm{estim}$, $d_\mathrm{de}^\mathrm{true}$ and to analyze the transformed values with the familiar linear methods (pearson correlation, additive contrasts, linear regression, ANOVA, etc.).

While I wrote that the the time model is not additive, in TTG experiment there are actually plausible reasons to use a model with an additive shift parameter. Namely, the observers probably consider

$$t_\mathrm{ttg}^\mathrm{estim}=t_\mathrm{walk}^\mathrm{estim}+\frac{d_\mathrm{car}^\mathrm{estim}}{v_\mathrm{car}^\mathrm{estim}}$$

That is, the time is the sum of the time until the vehicle arrives plus the time to cross the street. Unless the time to cross the street is negligable, $t_\mathrm{walk}^\mathrm{estim}$ should be included as a shift parameter in the model of the time estimates from the TTG experiment.

$\endgroup$
3
  • $\begingroup$ Thank for the effort you took to reply! But I think it's very difficult to make the assumption of $v_\mathrm{ttg}^\mathrm{estim}=v_\mathrm{ttg}^\mathrm{true}$. $\endgroup$ Commented Apr 18, 2017 at 8:25
  • $\begingroup$ Also the conversion of $v_\mathrm{ttg}^\mathrm{estim}$ into a $d_\mathrm{ttg}^\mathrm{estim}$ would not be easily comparaple to $d_\mathrm{de}^\mathrm{estim}$, I guess. What's indeed possible, is to do a linear regression for each proband and check the deviation of $d_\mathrm{de}^\mathrm{estim}$ from the regression, what do you guys think? $\endgroup$ Commented Apr 18, 2017 at 8:32
  • 1
    $\begingroup$ I would not compare $d_\mathrm{ttg}^\mathrm{estim}$ and $d_\mathrm{de}^\mathrm{estim}$. A simple approach would be to concatenate $\log d_\mathrm{ttg}^\mathrm{estim}$ and $\log d_\mathrm{de}^\mathrm{estim}$ to form vector $y$ and to concatenate $\log d_\mathrm{ttg}^\mathrm{true}$ and $\log d_\mathrm{de}^\mathrm{true}$ to form vector $x$ and then to estimate $y=x+\alpha+\epsilon$ (where $\epsilon$ is residual). $(e^\alpha-1)*100$ will tell You by how many percent people underestimate/overestimate the true distance. You may then add interactions for experiment and VR condition. $\endgroup$
    – matus
    Commented Apr 18, 2017 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.