I have two processes that output n-square matrices. I am interest in modelling these matrix norms as a dependent variable, according to the process used, and the matrix size n as predictors.
I generate those matrices relying on some random process that probably implies its own mathematical rules, but the aim here is more about modelling what happens from the obtained data.
Here are a small version of my larger dataset, in .txt, that should be importable through
So my aim is to have a model following something like
norm ~ process * n.
By design I also have "blocks" (5) in each of which I apply both processes (
P2) in several "trials" (5), each outputting a matrix norm (25 norms for
P1 and 25 for
P2, for a given
n). In other word, I feel I can model this as a nested random effect structure, such as:
norm ~ process * n + (1|Block/trial)
From data visualization
norm = f(n) I feel that you have a similarly shaped increase, with one increasing more rapidly.
ggplot(data, aes(n,norm,colour=process))+geom_jitter(alpha=0.2)+ geom_smooth(method="loess")
And the difference seems non significant with low
n, but the two curves part from each other as
n increases, and that's something I'm interested in. So I want a type of model that translates it into estimates and confidence values.
And that's the thing: it doesn't straight look like anything I tried to model before, especially with the slight S-shaped but unbounded (?) trend.
I tried fitting a LMM with glmmTMB or even MCMCglmm, with or without scaling of the predictors and/or response variable, but I end up with a very visible trend in the residuals against n, because of the S-shape of course.
I'm getting confused about what should be my approach:
is there a family that will "fit" for this job? What would be the data/design justification (in terms of type of response/heteroscedasticity or else)?
will this work better if I apply a reasonable transformation, e.g. dividing the norm in regard to
nor a function of it? scaling in relation to
n, the blocks? Another transformation that might "normalize" the relation between predictors and response?
is this a case where it'd be best to go into a non-linear regression or a GAM(M)? I never fitted anything similar, and from my reading I feel the smoothing function could help here. But that it might be an overkill for such a case...