1
$\begingroup$

I have some data as tuples $ (y,x)$. I am trying to fit a quadratic curve to the data, its known from the physics of the problem that the relationship should be quadratic. The problem is that I have isolated sets of experiments with other out of control confounding factors, that $y$ is a function of but are latent or unobservable.

Experiment 1 was run under conditions that could alter the value of $y$, but we are only measuring $x$ in this experiment, so we have pairs $x,y$

Experiment 2 was run under a different set of conditions that could alter the value of $y$, but we are only measuring $x$ in this experiment, so we have pairs $x,y$

And so on,

My question is what may be a robust way to fit the quadratic to the data, I have been reading about splines, loess regression etc but not sure if either of that can be used here. Please help.

Improve this question
$\endgroup$
4
  • $\begingroup$ In what way do you expect the experiments to alter $y$? Will it add a constant? Add a random amount? Change the relationship between $x$ and $y$? $\endgroup$
    – Ben Kuhn
    Commented Dec 8, 2014 at 21:58
  • $\begingroup$ The relationship does not change and a random amount is added. I guess the question is if we can do better than a simple lstsq by somehow clustering data for similar experiments. $\endgroup$
    – gbh.
    Commented Dec 8, 2014 at 22:07
  • $\begingroup$ Exactly what is this "relationship"? Is it supposed to be of the form $y=ax^2+bx+c$, $x=ay^2+by+c$, $ax^2+bxy+cy^2+dx+ey+f=0$, or perhaps something else? Which data values are subject to random variation? $\endgroup$
    – whuber ♦
    Commented Dec 8, 2014 at 23:05
  • $\begingroup$ Ben's approach seems very sensible. You might consider treating the experiment-effect as a random effect (random-intercept) in a mixed effects model. $\endgroup$
    – Glen_b
    Commented Dec 9, 2014 at 1:06

1 Answer 1

Reset to default
1
$\begingroup$

If you think that each experiment adds a random constant to $y$ (where the distribution of the random constant depends on the experiment), then you can simply include a quadratic factor for $x$ and code each experiment as a dummy variable (so your tuples go from $(y, x)$ to $(y, x, x^2, e_1, e_2, \dots)$, where $e_i = 1$ if the tuple is from experiment 1, and 0 otherwise). Then you can fit a standard linear model to find the parameters.

This corresponds to a model $y = ax^2 + bx + c_i + \epsilon$--so the constant that gets added to $y$ depends on which experiment you're in. This model assumes that the noise term $\epsilon$ is constant across all experiments (that is, the "experiment component" of the randomness can have a different mean across experiments, but has the same variance). If you want to check this assumption, you can plot the residuals grouped by experiment and check if the distributions look the same.

Improve this answer
$\endgroup$
7
  • $\begingroup$ The idea is interesting and thanks for the answer, but somehow I don't like the idea of using experiment as a variable. Part of the reason is that while building a prediction, we will not have access to that. Is there another way to combine the data in an intelligent manner? Robust regression maybe? Also another issue is that some experiments may only have a single x,y pair $\endgroup$
    – gbh.
    Commented Dec 8, 2014 at 22:15
  • $\begingroup$ When predicting, will people also be using experiments that alter the value of $y$? If so, do you know anything about the distribution of the experiments they'll be using? Will they be using experiments that aren't in the training set at all? Do you know anything about the distribution of the $c_i$? $\endgroup$
    – Ben Kuhn
    Commented Dec 8, 2014 at 22:19
  • $\begingroup$ No, no. no and no :) In general the idea is to have a holistic model without the notion of experiments. $\endgroup$
    – gbh.
    Commented Dec 8, 2014 at 22:35
  • $\begingroup$ Do you have any tuples $(y, x)$ where no experiment is being run? $\endgroup$
    – Ben Kuhn
    Commented Dec 8, 2014 at 23:11
  • $\begingroup$ You have a small sample for that, but the logic of linear mixed models would fit here: you would consider experiments as a random variable $N(0, \sigma)$, i.e. as a random sample of different possible conditions that could be encountered (so $c_i$ in @BenKuhn model would be a random effect). Notice also that you expect two things that rule out each other: (a) to include the impact of the experiment into the model, (b) not to include impact of the experiment in the model. $\endgroup$
    – Tim
    Commented Dec 9, 2014 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.