I have a nonlinear model: $$r10^{\beta_0-\beta_1A-\beta_2B-\beta_4D-\beta_{2,4}BD} $$. I used sas to test different models. Here is the code:

proc nlin data=bcoesdata;
     parameters r=10 j=5 k=5 l=1 m=1 p=1;
     model y = r*(10**(p-(j*a+k*b+l*d+m*b_d)));

After evaluating simpler models (linear regression did not fit), this one gave the following result:

enter image description here

Is the column Approx Pr > F indicating that the model is good since its value is lower than $0.01$? I never worked with nonlinear models before so I do not know what metric is appropriate and I guess that the right metric (test) for goodness of fit probably depends on the function used as a model . However, if there is a way to get a linear model that I have missed, feel free to answer that.

P.S. Data aren't normal, if that counts.

EDIT. Raw data can be downloaded from this link. Its format is as follows:

A B C D pid Y

where A, B, C, and D denote four parameters, pid is the id of a unit, and Y is the response variable. Data correspond to a full $2^k$ design. Low and high levels of the parameters are coded with -1 and 1. Each combination of A, B, C, and D values was evaluated on every unit in the sample.

Since the data are not normal, and I could not transform it to normality, I used Friedman's test to determine if changing the level of effect makes significant difference in Y. A, B, D and AD proved to be significant.

EDIT 2: The original question was how to determine goodness-of-fit of a nonlinear model from the beginning.

  • $\begingroup$ By "linear regression did not fit", how did it not fit? Was that linear model based on the logs of the dependent variable with $log_{10}(r)+\beta_0-\beta_1 A-\beta_2 B-\beta_{2,4} BD$ ? $\endgroup$ – JimB Apr 8 '16 at 17:01
  • $\begingroup$ I got it with proc reg. Its R-square was 0.0169 , adjusted R-square was 0.0031 and Pr > F column was 0.3026. proc reg should inspect the model of the form $\beta_0+\beta_1A+\beta_2B+\beta_{2,4}BD$. $\endgroup$ – Milos Apr 8 '16 at 17:14
  • $\begingroup$ Yes, but did you take the log of the dependent variable? If you could share your data, that would help pinpoint the issue. You could also construct an R-square from the table you present and that R-square doesn't look very large. $\endgroup$ – JimB Apr 8 '16 at 17:18
  • $\begingroup$ If I transform the response variable, $y$, to $z=log_{10}{y}$, proc reg returns $0.2534$ for R-squared, $0.2428$ for adjusted R-squared and $<0.0001$ for Pr>F column. So, I inverted this and came up with $10^{...}$ model. I did not expect R-square to be near $1$, so lower values should not represent a big problem. $\endgroup$ – Milos Apr 8 '16 at 17:21
  • $\begingroup$ I can share the raw data, but later, when I get home. :) $\endgroup$ – Milos Apr 8 '16 at 17:23

Thanks for sharing the data. Because this is a $2^k$ factorial and there appear to be 18 replications (identified by pid), I would treat this as a mixed model and use PROC MIXED (as you're using SAS) using the log of the y values:

ods graphics on;
proc mixed data=test covtest;
  class a b d pid;
  model logy = a|b|c|d / solution residual;
  random pid;
  repeated / group=d;
ods graphics off;

This analyzes the experiment as it was laid out with the addition of estimating separate residual variances for each level of d. (A more complicated variance-covariance structure is likely warranted which I'll touch on later.)

This gives a fixed effect for every combination of the 16 combinations of a, b, c, and d. (I suspect that you mentioned only a, b, d, and b*d because a previous analysis suggested that these were the only significant effects. I'm pretty conservative about these things and would argue that you need to do the analysis as designed rather than performing any variable selection on such designed experiments.)

Here are the tests of each of the 16 combinations:

Tests of fixed effects

and estimates of variance components

Covariance parameter estimates

and a set of residual plots:

Studentized residual plots

In the plot of the studentized residuals vs. the predicted values, there are neighboring pairs of predicted values that seem to have matching residuals which strongly suggests that there's more to the design (either a timing or spatial relationship) than just the sharing of a pid. (This is why some of the "columns" of residual points look a little fuzzy as there are two sets of points that are just slightly offset from each other.) If there is more to the experimental design than just pid, then I'd modify the PROC MIXED statements to account for that.

Estimates of the fixed effects can be obtained using the LSMEANS statement.

Update: I really didn't answer your specific question because I don't see that a nonlinear model is necessary. As a reminder when one is looking for normality, it's not about the dependent variable being approximately normal but rather the residuals. It seems that a linear model using the log of the dependent variable gives an adequate description of your data (other than maybe what appears to be the additional covariance structure). One can obtain the LSMEANS and transform back to the original units.

Additional update

Here is a figure showing why I think that there's something else going on with the data. For each of the 16 combinations of a, b, c, and d for each pid I've connected those points. There seems to me to be additional patterns to be explained. There at least one group of pid's that reacts differently than the others.

Patters in the pids

| cite | improve this answer | |
  • $\begingroup$ Thank you for your answer. I have just one more question. Why did you use d in the repeated statement? $\endgroup$ – Milos Apr 9 '16 at 17:27
  • $\begingroup$ By the way, you are right about the pids. There are three groups of units and I will include this information in the analysis. $\endgroup$ – Milos Apr 9 '16 at 17:38
  • 1
    $\begingroup$ I used group = d because in looking at the simple standard deviations of the 16 combinations, the biggest separation in standard deviations was associated with the two levels of d. The option "group = d" results in two different error variances. This doesn't usually change the cell means much at all but can make a big difference in standard errors of effects and/or tests of hypotheses. $\endgroup$ – JimB Apr 9 '16 at 23:41
  • $\begingroup$ Thank you a lot! :) If you can recommend textbook(s) where modeling and other statistical tests and techniques are explained well, I will be extremely grateful. :) I have some background in statistics, but it just wasn't enough to handle this properly. $\endgroup$ – Milos Apr 9 '16 at 23:51
  • $\begingroup$ My first recommendation would be to get the advice of a statistician. In terms of books about fitting models with SAS (especially models with random effects) you should consider SAS for Mixed Models. $\endgroup$ – JimB Apr 10 '16 at 3:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.