Probably what you will need to use is the Parametric Bootstrap Cross-fitting Method. Here is the basic procedure:
- Fit each model to the data. Estimate the models' parameters and extract your favorite measure of goodness of fit. We will call the model with the higher value for this GoF measure $A$ and the other model $B$. Calculate the difference $d$ between the two measures of GoF, and store that value. (Be sure you are clear about whether higher or lower numbers of your GoF measure indicate a better fit--i.e., whether $d>0$ implies $A$ is better or worse.)
- Using the fitted parameters for $A$ from step 1, generate a large number of synthetic datasets (say 1000). With each of these datasets, fit both of your models, extract their GoF measures, compute $d$ and store it.
- Using the fitted parameters for $B$ from step 1, generate another set of (1000) synthetic datasets. With these datasets, again fit your models, compute the $d$s and store them.
- You now know what the sampling distribution of $d$ looks like when the true model is $A$ and when the true model is $B$. Determine the cutpoint, $d_\text{cut}$, that optimally differentiates between the models. If you want, you can bring prior knowledge to bear by differentially weighting the alternatives.
- Compare your found $d$ from step 1 to $d_\text{cut}$ and select the corresponding model.
I demonstrate this approach here. (There is another description of PBCM in this answer: Measures of model complexity) Here is the reference:
- Wagonmakers, E.J., Ratcliff, R., Gomez, P., & Iverson, G.J. (2004). Assessing model mimicry using the parametric bootstrap, Journal of Mathematical Psychology, 48, pp. 28-50. (pdf)
