I'm working on a nonlinear optimization problem (using the L-M algorithm) and I've been reading about badly-scaled problems, in which parameters vary on orders of magnitude so they must be rescaled accordingly for better convergence. This comes from the fact that most deterministic optimization techniques use the gradient to determine the search direction, therefore badly scaled parameters will lead to badly scaled Jacobians, thus compromising the search.
After reading this, I started to wonder what would be the influence of functions with different output scales on the optimization. For example, if I want to find A and B that will fit F1(A,B) and F2(A,B) to some data, with F1 output values being on average much greater than F2, wouldn't the optimization be biased towards fitting only F1?
I know from machine learning that you should always normalize or standardize the data (input and output), but I cannot change the inputs in my case, since F1 and F2 are not black-box functions and are very nonlinear.
Should I normalize or standardize only the outputs of F1 and F2? Any literature recommendation on this would be appreciated.
Thanks.
