1
$\begingroup$

I am struggling to write a simple proof for the following statement:

The neuron's inputs are proportional to the probability of the respective feature in the input layer

In particular, I am not sure what $y'$ means here and how we get to an expression of $p(\mathbb{y}|x)$ that depends on $p(\mathbb{x}|y)/p(\mathbb{x}|y')$. Also not 100% sure what $n$ means.

source: https://arxiv.org/pdf/1503.02406.pdf

EDIT 1 I found this question to be a duplicate of this older question. There, the author of the referenced paper points the reader in the right direction.

EDIT 2 In light of the answer provided, I feel that the only missing piece of the puzzle is equation (2) - below.

enter image description here

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ I assume you mean p(y|x) in terms of ... $\endgroup$ – gunes Apr 2 '19 at 14:57
  • 1
    $\begingroup$ I agree, it's a bit awkward to use n as proportionality constant. $\endgroup$ – Tom Apr 2 '19 at 15:21
  • $\begingroup$ In another thread (added link to Q, see Edit 1), the author of the referred paper says that $n$ is the sample size and points to some sort of Signal to Noise ratio and the assumption that the SNR grows like $\sqrt{n}$. I still don't get where eq (2) comes from: although I get the conditional independence part, I don't understand the $n p(x_j)$ exponent. $\endgroup$ – IcannotFixThis Apr 3 '19 at 8:31
2
$\begingroup$

$y'$ is the other class, i.e. y-not. If you get rid of the log and exp in the denominator of Eq. (1), you'll have: $$p(y|x)=\frac{1}{1+\exp\left(\log\left(\frac{p(x,y')}{p(x,y)}\right)\right)}=\frac{p(x,y)}{p(x,y)+p(x,y')}$$ The denominator needs to be $p(x)$ due to Bayes Rule, so $y'$ is really the negative class. $n$ is just a proportionality constant, which is also related to your yellow-quoted line. i.e. they actually mean something like $h_j\propto p(x_j)$.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.