4
$\begingroup$

Should the weights of a neural network without hidden layer and a logistic activation function be the same as the parameters of a logistic regression? Mine are not the same?

nnallnohidden=nnet(
    PartialPrepayzo~FIXPER+MEDSAL2+DREL+LEEFTIJD+HH2CRED+LTV_curr+
    rate1Y+rate5Y+CIremFIRP+URB+WELSTAN2+OutNot+mover+SavRate+CRate,
    data=test,
    size=0,
    skip=T)

log <- glm(PartialPrepayzo~FIXPER+MEDSAL2+DREL+
LEEFTIJD+HH2CRED+LTV_curr+rate1Y+rate5Y+CIremFIRP+URB+WELSTAN2+OutNot+mover+SavRate+CRate, data = test, family = "binomial")
summary(log)

                 [,1]           [,2]
 [1,] -1.029560622391 -1.26664018566
 [2,] -0.078225500455 -0.06536644222
 [3,]  0.410455341173  0.67036107254
 [4,]  0.006961510972 -0.11463794856
 [5,]  0.473629162069  0.70074482878
 [6,]  0.614550199698  0.83536187570
 [7,] -0.612837570442 -0.48086112696
 [8,] -0.743739495966 -1.06994471577
 [9,]  0.200419240204  0.83957097597
[10,] -0.166568966328 -0.50583277715
[11,]  0.017640270701  0.12678131085
[12,] -0.005947704128 -0.04248886193
[13,] -0.428175932694 -1.69521649738
[14,]  0.049657239050  0.26482261363
[15,]  1.602200661890  2.50479250068
[16,]  0.367771764513  0.96127873663
$\endgroup$
  • 1
    $\begingroup$ No shouting in titles please. $\endgroup$ – Nick Cox Aug 20 '13 at 12:09
  • 3
    $\begingroup$ nnet() by default optimizes squared error instead of binomial loss. Try using entropy=TRUE to have it optimize the same loss function as logistic regression. $\endgroup$ – Shea Parkes Aug 20 '13 at 13:08
  • 1
    $\begingroup$ It also looks like you have a fair number of features; I'd try something like an elastic net penalty from the glmnet package to see if any of those fall out. You can do a pure ridge penalty in nnet() using the weight parameter (but would have to optimize it via resampling). $\endgroup$ – Shea Parkes Aug 20 '13 at 13:10
  • 4
    $\begingroup$ If you have lost your login credentials, please follow instructions in our Help Center to merge your different accounts. Sidenote: It is a bad idea to rollback changes that are made by benevolent users who are just willing to help and improve your post. $\endgroup$ – chl Aug 20 '13 at 16:23
2
$\begingroup$

Reproducing comments:

  • Shea Parkes: nnet() by default optimizes squared error instead of binomial loss. Try using entropy=TRUE to have it optimize the same loss function as logistic regression.

Also, to reproduce a logistic regression, you need the input to output activations to be linear, but the readout layer to predict probabilities. This is because the logistic regression is estimating a linear function of the inputs. The readout layer piece is accomplished by Shea’s comment, but you’ll also need to adjust the internal activations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.