Coefficient for linear and non-linear regression I have used a deep NN for performing regression analysis with multiple independent variables and then predicting one dependent varible.
To understand the quality of the regression I have used $R^2$, but it is typically used for linear regression.
My question is, Can I use $R^2$ coefficient for determining the quality of such regression. Please take into account that the problem I'm focusing on should be non-linear. If no, which would be the corrent coefficient, instead of $R^2$, in case of non-linear regression.
Thank you in advance
 A: R2 can be used. Also you can check all the loss functions used in regression settings, such as MSE (mean squares error) MAE (mean absolute error) etc.
A: "Linearity" is not an issue here. You can most likely interpret your regression as a linear regression over non-linearly transformed variable. What matters is your loss function. If you're minimising the sum of squared errors, $R^2$ is the perfect measure of performance.
I disagree with Haitao: $MSE$ is redundant with $R^2$: $MSE = 0 \Leftrightarrow R^2 = 1$ and $MSE = Var(y) \Leftrightarrow R^2 = 0$. Mean absolute error is likely to be correlated, but a less suitable measure than $R^2$ -- unless, of course, your regression is minimising the sum of absolute errors.
A: I wouldn't really use R2 for NN comparisons.  I would instead use the name of the forum and look at your cross validation MSE (you could use cross validation R2 but I would just use the raw MSE) or RMSE.  The issue is that you can, with NNs or Random Forests or other non-parametric models, get inflated R2 values.
The 'best' NN then is the one which minimizes your average MSE values across all test folds.  So I wouldn't say that a model is a 'good' model or not, just that it is the 'best' or most 'useful' of the ones you tried.
And even for a linear model, a model with a R2 of .95 is, in my experience, still a 50/50 if it is any better than a model of the same data with an R2 of .9 so I would never really say a model is good based off of it alone.
