# Difference between simultaneous fitting and separate fitting

I get from a specific experiment 3 outputs. Each is the information of some physical quantity in the direction x,y or z. The way we extract the information from those signals is by fitting.

In the data extracted from fitting, some parameters depend only on the direction of the physical quantity we want to measure, and some others depend on the whole system or on the scalar value of the physical quantity (common attributes). Statistically, it's preferred to do a simultaneous fit for all the directions. This is what my boss told me.

What he said sounds right, but technically, it's pretty hard to fit the 3 data sets together, because the function is already pretty complicated with 7 parameters in each direction, and combining the 3 together will just complicate stuff, and my fit doesn't converge when doing them simultaneously. I'm using Levenberg-Marquardt's algorithm.

The question is, is there anything I could do in the individual results of the fit, to obtain the result of a simultaneous fit? In other words, how can I avoid simultaneous fitting and obtain its statistical advantage?

And if simultaneous fitting is the only solution I have, what does taking 3 data sets to a simultaneous fit entail? how will that scale the fitting parameters (Chi^2, weighting, tolerance ...etc)?

I'm fitting using a program I wrote myself with C++.

Thank you for any efforts.

-
The danger of optimizing your parameters sequentially is to miss the global optimum, depending ont what you fit and how, this will be an issue or not. I fear there is no general answer. Your last question is also very general and does not have a general answer as far as I know. –  gui11aume Jun 2 '12 at 11:32
Well, I mean with my last question that there has to be some kind of dependence between the Chi^2 and the other parameters from the individual fits to the simultaneous fits. Like for example when we say that the weighting has to be 1/sigma^2, and it could be probably independent of the number of simultaneous fits we're doing. My whole problem is that the same program that fits for each individual signal doesn't work for simultaneous fitting. So I was wondering what kind of changes have to be done to the program in order to accept simultaneous fitting?! –  Samer Afach Jun 2 '12 at 11:51
Are the three quantities you call X, Y and X measured in the same units? If not, simultaneous fitting would require somehow normalizing those values. Minimizing the sum (for the three variables) of the sum (over all points) of squares only makes sense when the three variable are in the same units, or normalized to be so. –  Harvey Motulsky Jun 2 '12 at 14:46
Yes they have the same units. I didn't really get how this would help answer my question. Could you please explain? –  Samer Afach Jun 2 '12 at 21:57

Good advice, but doesn't this presuppose some linear structure? A related point is that the dataset appears to be multivariate: each record, if I interpret the question correctly, will contain a vector of experimental settings and a three-vector $(x,y,z)$ of responses ("output"). To which of these do you propose applying PCA: the settings vector, the output vector, or the combination? –  whuber Jun 6 '12 at 13:06