1
$\begingroup$

I have data for $30$ subjects completing a task, each subject completed the task $4$ times. The only data I have is a single value per session quantifying some property of how the subjects performed in the task. This value is between $0$ and $1$. I am interested in comparing the means of the first two completions to the last two over all subjects. For reference, the normalised data is distributed as follows.

enter image description here

My two concerns are that the data doesn't look especially normally distributed and that the samples in the first and second group may not all be independent.

$\endgroup$
6
  • $\begingroup$ Sorry, but this is not very clear. How do you only have four samples? Do you mean observations? Or are you taking separate samples of each variable? Then are you taking means across variables? That doesn't sound sensible. Later it seems like you have 30 observations on four variables. Also, you say the data in the graph are normalized, but clearly they aren't. It would be good, I think, if you describe your actual case, in non-statistcal language. $\endgroup$
    – Peter Flom
    Commented Apr 6 at 10:48
  • $\begingroup$ @PeterFlom thank you for your comment - I can see that this was not well worded. I will edit this now. $\endgroup$
    – mrepic1123
    Commented Apr 6 at 10:53
  • $\begingroup$ @PeterFlom I hope that is more clear now. $\endgroup$
    – mrepic1123
    Commented Apr 6 at 10:57
  • $\begingroup$ Yes, it is. Thanks. $\endgroup$
    – Peter Flom
    Commented Apr 6 at 10:58
  • 1
    $\begingroup$ A simple approach is to calculate, for every subject, Post minus Pre, where Pre is the average of the first two values, and Post is the average of the last two values. Then you can use any appropriate one sample test. $\endgroup$
    – Michael M
    Commented Apr 6 at 20:31

1 Answer 1

0
$\begingroup$

Unless you have a good reason, e.g. something like data is bi-modal, I would suggest modelling your task outcome as either beta or logit-normal distributed. e.g. something that respects form of your data. Normalizing the results, I would say, is a bad idea here.

If you want to compare two measurements of such variables (two earlier completetions vs two later completions), you can consider difference between them, but it would make more sense to me to compare the ratios, since a product of two numbers between 0..1 stays within the same range, the same cannot be said about differences.

So one way to go here may be to pose a model as follows:

$$ \begin{align} \mu_i &\sim Normal\left(A,B\right)\\ \sigma_i^2 &\sim LogNormal\left(C,D\right) \\ r &\sim Beta\left(\alpha,\beta\right)\\ \eta_i^{(1,2)} & \sim LogitNormal\left(\mu_i,\sigma_i^2\right) \\ \eta_i^{(3,4)}/r & \sim LogitNormal\left(\mu_i,\sigma_i^2\right) \end{align} $$

So you model the first two task outcomes for $i$-th person ($\eta_i^{(1,2)}$) as coming from LogitNormal distribution with some person specific $\mu_i$ and $\sigma^2_i$, which in turn come from common distributions parametrized by $A,B,C,D$. If this is an overkill you can simply have single $\mu$ and $\sigma^2$ for all individuals. The difference between the outcome of the first two tasks and the last two tasks is extracted as the ratio $r$ which is modeled as a beta-distributed value. If you do want to stick with differences in the means, you could have something like:

$$ \begin{align} \nu &\sim Normal(E,F) \\ \eta_i^{(3,4)}/r &\sim LogitNormal\left(\mu_i+\nu,\sigma_i^2\right) \end{align} $$

Either way, you end up fitting up to six constants to your data. This can be done in multiple ways, e.g. with MLE. I wrote the whole thing like a Bayesian problem, so that is also an option.

You wanted a test though. What would the null hypothesis be? That $r=1$ or $\nu=0$? If you estimate the distributions for these parameters it will be fairly straight-forward. Alternatively, something like Likelihood-ratio test could be attempted. It might be a good idea to do some simulations to understand what size of effect are you powered up to detect.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.