How to fit a set of curves having some free and some shared parameters?

Question

The problem

I have a data set with 1 dependent and $N$ independent variables $x_1,\dots ,x_N$ (all real numbers), and need to fit a (nonlinear) function/curve $$ f_i(x_i; p_1,\dots ,p_{k_F}, r_1, \dots, r_{k_S}) $$ to each independent variable $x_i$. The curves depend on two kinds of parameters: the parameters $p_1,\dots ,p_{k_F}$ are free to vary from one curve to the other, whereas the parameters $r_1, \dots ,r_{k_S}$ must be shared among the curves (but their optimal values are not known a priori). The goal is to find the point $\langle p_1,\dots ,p_{k_F},r_1, \dots ,r_{k_S}\rangle$ in the parameter space that minimizes an objective function (in my specific application, this can be e.g. the cumulative sum of squared residuals $y_i - f_i(x_i)$ across all the $N$ curves, where $y_i$ is the empirical value of the dependent variable in the data set).

The main question

Is there a generally accepted optimal strategy for solving such an optimization problem, where some parameters are free to vary but others must have the same value across curves?

What I've tried so far

This is not my field of specialism, but I have toyed with some ideas. The best I've been able to come up with so far is the following procedure:

1. Define sets of values that the shared parameters $r_1, \dots ,r_{k_S}$ can assume.
2. Loop over these sets: for each combination of shared parameter values, find the best-fitting free parameters $p_1, \dots ,p_{k_F}$ using some off-the-shelf method such as nonlinear least squares. Record the value of the objective function and said best-fitting free parameters.
3. Having looped through the shared parameter value sets, return the combination of $r_1, \dots ,r_{k_S}$ and $p_1, \dots ,p_{r_F}$ that gave the best fit.

This method works, but it has the obvious drawback that it doesn't scale well as the number of shared parameters $k_S$ increases. Furthermore, it is not always clear how the value sets to be probed in the loops ought to be defined (i.e. how to set the value range endpoints and resolution). I also find it problematic (though can't quite articulate why) that the shared parameters are treated differently from the free ones, i.e. the shared parameter values are searched from finite, ad hoc defined sets, whereas the free parameters are allowed to vary to floating-point precision (depending on what sort of fitting algorithm is used to fit the free parameters, of course).

The procedure can be refined a bit by embedding it in a further loop that makes the parameter value ranges successively smaller with each iteration, thereby kind of "zooming in" on the optimal parameter value combination. But this is just an embellishment.

The other question

In case the above procedure is actually the best way of tackling the problem (which I very much doubt), does it have a name?

Answer 1

One possibility that should scale well is to: A) Independently fit each data set, then B) Determine the mean (or median) value of the independently fitted values for what should be the shared parameter, and finally C) re-fit the individual data sets using the mean value as a constant that is not fitted. Note that if the individual data sets have different numbers of data points the mean value from step B will effectively be weighted toward the data sets with the most data, if this is not desired then a weighted mean should be used with the weighting dependent on the number of data points per data set. This process should be easily automated.