If I use radial basis function networks (RBFNs) for probability estimation by plugging the output of the RBFNs into the Logistic function are weights between 0 and 1 sufficient?


No, if the weights are bounded then there will be a bound on the log odds ratio going into the logit function, and hence a bound on the range of probabilities that the model can predict. However in practice to get a 1 from the logit function you don't need an infinite log odds ratio, because of numerical precision issues.

I would imagine that positive weights is O.K., provided that the bias parameter is allowed to be negative.

However, if you want to estimate probabilities, you want to use something like kernel logistic regression, or build the logit into the training procedure rather than just tacking it onto a trained RBF network, see this paper by Ian Nabney for details.

Appologies if I have not understood the question properly.

  • 1
    $\begingroup$ I'm a bit confused. What the asker is doing, I think, is some kind of hierarchical model. First, they use RBFN to estimate some kind of network, then they use some kind of fitted values from that network as inputs to a ordinary logistic regression model. If the input to the logistic regression is bounded by {0,1}, it shouldn't strike us as weird. The Pearson Chi-Square test for 2 by 2 tables is just a score test for a logistic regression model adjusting for 0/1 exposure levels. Because the input is literally interpreted as a probability, we might scale the log-odds ratio for interpretation. $\endgroup$ – AdamO Dec 20 '11 at 18:23
  • 1
    $\begingroup$ @Adam I am using an algorithm that adapts the parameters and structure of a RBFN and plugs its outputs into the en.wikipedia.org/wiki/Logistic_function to obtain probabilities whilst optimising the cross entropy. Looking at the function it makes sense to use weights between at least -6 and 6 to easily obtain probabilities between 0 and 1 without having to use several similar basis functions at the same place. $\endgroup$ – cs0815 Dec 21 '11 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.