4
$\begingroup$

Should the weights of a neural network without hidden layer and a logistic activation function be the same as the parameters of a logistic regression? Mine are not the same?

nnallnohidden=nnet(
    PartialPrepayzo~FIXPER+MEDSAL2+DREL+LEEFTIJD+HH2CRED+LTV_curr+
    rate1Y+rate5Y+CIremFIRP+URB+WELSTAN2+OutNot+mover+SavRate+CRate,
    data=test,
    size=0,
    skip=T)

log <- glm(PartialPrepayzo~FIXPER+MEDSAL2+DREL+
LEEFTIJD+HH2CRED+LTV_curr+rate1Y+rate5Y+CIremFIRP+URB+WELSTAN2+OutNot+mover+SavRate+CRate, data = test, family = "binomial")
summary(log)

                 [,1]           [,2]
 [1,] -1.029560622391 -1.26664018566
 [2,] -0.078225500455 -0.06536644222
 [3,]  0.410455341173  0.67036107254
 [4,]  0.006961510972 -0.11463794856
 [5,]  0.473629162069  0.70074482878
 [6,]  0.614550199698  0.83536187570
 [7,] -0.612837570442 -0.48086112696
 [8,] -0.743739495966 -1.06994471577
 [9,]  0.200419240204  0.83957097597
[10,] -0.166568966328 -0.50583277715
[11,]  0.017640270701  0.12678131085
[12,] -0.005947704128 -0.04248886193
[13,] -0.428175932694 -1.69521649738
[14,]  0.049657239050  0.26482261363
[15,]  1.602200661890  2.50479250068
[16,]  0.367771764513  0.96127873663
Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ No shouting in titles please. $\endgroup$
    – Nick Cox
    Commented Aug 20, 2013 at 12:09
  • 3
    $\begingroup$ nnet() by default optimizes squared error instead of binomial loss. Try using entropy=TRUE to have it optimize the same loss function as logistic regression. $\endgroup$
    – Shea Parkes
    Commented Aug 20, 2013 at 13:08
  • 1
    $\begingroup$ It also looks like you have a fair number of features; I'd try something like an elastic net penalty from the glmnet package to see if any of those fall out. You can do a pure ridge penalty in nnet() using the weight parameter (but would have to optimize it via resampling). $\endgroup$
    – Shea Parkes
    Commented Aug 20, 2013 at 13:10
  • 4
    $\begingroup$ If you have lost your login credentials, please follow instructions in our Help Center to merge your different accounts. Sidenote: It is a bad idea to rollback changes that are made by benevolent users who are just willing to help and improve your post. $\endgroup$
    – chl
    Commented Aug 20, 2013 at 16:23

1 Answer 1

Reset to default
2
$\begingroup$

Reproducing comments:

  • Shea Parkes: nnet() by default optimizes squared error instead of binomial loss. Try using entropy=TRUE to have it optimize the same loss function as logistic regression.

Also, to reproduce a logistic regression, you need the input to output activations to be linear, but the readout layer to predict probabilities. This is because the logistic regression is estimating a linear function of the inputs. The readout layer piece is accomplished by Shea’s comment, but you’ll also need to adjust the internal activations.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.