1
$\begingroup$

I am working with R with this code from the book "Bootstrap Methods: With Applications in R" by Gerhard Dikta and Marsel Scheer:

set.seed(123,kind ="Mersenne-Twister",normal.kind ="Inversion")
semiparametric_data <-
  data.frame(X = runif(400, min = 1, max = 30)) %>%
    dplyr::mutate(
    mu = 4 * exp(-X/2) - 3 * exp(-X/10), epsilon = rnorm(400, sd = 0.25),
    Y = mu + epsilon)

fit_sp <- minpack.lm::nlsLM(
  formula = Y  ̃ a * exp(X/b) + c * exp(X/d),
  data = semiparametric_data,
  start = c(a = 4, b = -2, c = -3, d = -10),
  control = nls.control(maxiter = 1000))
fit_sp

## Nonlinear regression model
  ##   model: Y  ̃ a * exp(X/b) + c * exp(X/d)
  ##    data: semiparametric_data
  ##      a      b      c      d
  ##  3.707 -2.105 -3.025 -9.797
  ##  residual sum-of-squares: 23.76
  ##
  ## Number of iterations to convergence: 3
  ## Achieved convergence tolerance: 1.49e-08
  1. What does minimize the nls function? RMSE or MSE?
  2. What is the difference between minimizing RMSE or MSE?

By a theoretical/mathematical point of view the resulting coefficients should be the same but in fact the resulting coefficients are slightly different.

  1. What is more efficient to minimize between RMSE and MSE in non linear least squares?
$\endgroup$
3
  • 1
    $\begingroup$ nls minimizes the sum of the squares of the residuals. $\endgroup$ Jan 3 at 23:47
  • $\begingroup$ @jdobres thank for your suggestion. I posted the question here because I thought the difference in coefficient estimates could be explained by the efficiency of the solver. From a statistical point of view there should be no difference between RMSE and MSE minimization. $\endgroup$
    – Barbab
    Jan 4 at 0:02
  • $\begingroup$ I'm not sure what coefficients you're comparing here that are "slightly different"; you only show a fit from minpack.lm::nlsLM. Are you comparing this to nls? As noted in my answer, nls and nlsLM use different algorithms (Gauss-Newton vs Levenberg-Marquardt). $\endgroup$
    – Ben Bolker
    Jan 4 at 1:03

1 Answer 1

1
$\begingroup$

There should be very little difference in the results between minimizing any reasonable monotonic, sign-preserving transformation of the sum of squares (if the sign were flipped we would need to maximize rather than minimizing): sum of squared residuals, the square root of the SSQ, the mean square (SSQ/n), the root mean square ... As you point out, mathematically/statistically there should be no difference. Computationally, the only differences are in floating point accuracy. I can imagine that RMSE could be slightly more accurate in the case where the scale of the response was extremely large or the range of scales of response was very large, but I would be surprised if the difference was noticeable in any practical context.

nls minimizes the sum of squared residuals; if you wanted to play with this you could try different objective functions in one of R's general-purpose minimizers (optim, nlm, nlminb, ...) - but the difference in efficiency between the various minimizers used by optim/nlm/nlminb, the more specialized ones used by nls (Gauss-Newton) and nlsLM (Levenberg-Marquardt), and even particular choices made when implementing these algorithms, are likely to make more difference than the choice of scale of the objective function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.