Goodness of fit for nonlinear model

Question

We have fitted a nonlinear function to observed data. The next step should be the assessment of the goodness of fit of this function (like $R^2$ for linear models).

What are the usual ways to measure this?

Edit 1:

The fitting was performed as follows:

1. Perform a linear regression with independent variables A and B.
2. Calculate distribution's parameters from regression parameters. (The distribution is nonlinear and has variable C as an input.)
3. Assess goodness of fit of nonlinear distribution by comparing estimated to observed data.

Edit 2:

Examples for the steps mentioned above:

1. Regression model: $log(y) = \beta_0 + \beta_1 \centerdot log(a) + \beta_2 \centerdot log(b)$
2. $\rho = -\frac{\beta_0}{\beta_1}$ and $\theta = \beta_2$ for the following nonlinear distribution: $f(a) = \rho \centerdot a^{-\theta}$
3. Assess the goodness of fit of $f(a)$ with a given set of $(a, f(a))$ observations.

Answer 1

There maybe more to it, but to me it seems that you just want to determine goodness-of-fit (GoF) for a function f(a), fitted to a particular data set (a, f(a)). So, the following only answers your third sub-question (I don't think the first and second are directly relevant to the third one).

Usually, GoF can be determined parametrically (if you know the distribution's function parameters) or non-parametrically (if you don't know them). While you may be able to figure out parameters for the function, as it appears to be exponential or gamma/Weibull (assuming that data is continuous). Nevertheless, I will proceed, as if you didn't know the parameters. In this case, it's a two-step process. First, you need to determine distribution parameters for your data set. Second, you perform a GoF test for the defined distribution. To avoid repeating myself, at this point I will refer you to my earlier answer to a related question, which contains some helpful details. Obviously, this answer can easily be applied to distributions, other than the one mentioned within.

In addition to GoF tests, mentioned there, you may consider another test - chi-square GoF test. Unlike K-S and A-D tests, which are applicable only to continuous distributions, chi-square GoF test is applicable to both discrete and continuous ones. Chi-square GoF test can be performed in R by using one of several packages: stats built-in package (function chisq.test()) and vcd package (function goodfit() - for discrete data only). More details are available in this document.

Answer 2

Well, in Machine Learning the thing called Cross Validation is performed pretty often for purpose of model testing (test if that type of a model with these hyper-parameters - like number of degrees of freedom or whatever - fits your problem) - you split your data several times into train and test data sets, then run optimization over training set and compute whatever quality over tests data. The most confidential way is to run so called "QxT-fold cross validation". The pseudocode could could like:

cv_values = []
for t in range(T):
    split = randomsplit(data, number_of_parst = Q)
    for test_id in range(Q):
        model.fit(split[:test_id] + split[test_id + 1:] # test on everything excepting test_id
        cv_values.append(model.test(split[test_id]))

cv_values.mean() # whatever