0
$\begingroup$

I want to calculate the difference between a certain value associated with a decision alternative and the value associated with the objectively correct alternative as a measure of decision accuracy in an experiment (e.g. predicted cashflow of certain assets; calculate difference between net cashflow of chosen asset and the asset with the highest net cashflow in the set of alternatives).

To get a standardized measure of accuracy as a dv for my analysis, I was thinking about z-transforming the difference between chosen alternative and objectively correct alternative based on the answers of the experiment's participants. Higher z-values in this case would mean that participants were closer to the correct choice.

My question is, whether this transformation is meaningful and still contains the relevant information about accuracy (i.e. distance betweeen participant's cohioce and 'correct' one. Further I would like to know whether I can use the z-scores in a subsequent multiple linear regression and/or ANOVA/ANCOVA as a dv.

My question is related to this one: regression with z-scores as composite variables? however, I am especially interested whether the z-score are senseful as an accuracy measure.

Thanks a lot!

$\endgroup$

1 Answer 1

1
$\begingroup$

I would very seldom get away from the raw measurements. z-scores make the interpretation much trickier and have hidden assumptions. First, they assume that the standard deviation is the "gold standard" dispersion measure. This assumes among other things symmetry of the distribution. Second, some z-scores involve a standardization procedure where one subjects an expected value in such a way that linearity is implicitly assumed.

Lastly, when you use z-scores in subsequent analyses you are hiding the uncertainty in the mean and standard deviation used to compute the z-score.

$\endgroup$
2
  • $\begingroup$ Thank you very much for your answer and your very valid pionts of criticism. The raw values might indeed be a good idea to use, as they also convey information about the degree to which a decision does not follow rational choice theory. I will try to use them for the analysis and see if I run into any other issues. It's good to know, however, that z-scores are not the value of choice in this case. $\endgroup$
    – statleo
    Jun 26, 2019 at 13:01
  • $\begingroup$ Sorry for the double comment, I was just wondering whether transforming the accuracy variable such that the lowest difference between the choice of the participant and correct choice (i.e. difference of zero) is equal to 1 and the highest difference between choice of the participant and correct choice is equal to 0. Then the differences of all the other choices could be expressed in relation to the maximum difference. Would such a measure still be meaningful and suited for the subsequent analyses? Thanks for your opinion! $\endgroup$
    – statleo
    Jun 26, 2019 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.