enter image description here

At the output of the final layer of yolo, a leaky-relu is applied to the output, so if we have negative values for the width and height, the cost function will return a null value since we would have square rooted a negative value at the second sum of the cost. Thus not able to update the weights using back prop.

Am I wrong about this or is there something I am missing here? If I am not wrong, how do we guarantee the width and height to be positive?

  • $\begingroup$ You are summing squared values; The sum will be at worst 0, assuming lambdas are positive. $\endgroup$ – Vladislavs Dovgalecs Jan 10 '18 at 8:25
  • $\begingroup$ @VladislavsDovgalecs Sorry I don't see how that solves my problem, the square root would have still returned an imaginary value if the inside is negative, and if I do have an object in the cell, the term is not gonna become zero. $\endgroup$ – Chester Cheng Jan 10 '18 at 8:33
  • $\begingroup$ You asked "how do we guarantee the output to be positive" and I understood it that way, how to prove the expression you wrote is positive. I apologize for having misunderstood your question. $\endgroup$ – Vladislavs Dovgalecs Jan 10 '18 at 19:11
  • $\begingroup$ @VladislavsDovgalecs Thanks for your response, I will edit my question $\endgroup$ – Chester Cheng Jan 11 '18 at 0:31

According to their source code, actually they use an exp operation to ensure $w$ and $h$ are non-negative values.

box get_region_box(float *x, float *biases, int n, int index, int i, int j, int w, int h, int stride)
    box b;
    b.x = (i + x[index + 0*stride]) / w;
    b.y = (j + x[index + 1*stride]) / h;
    b.w = exp(x[index + 2*stride]) * biases[2*n]   / w;
    b.h = exp(x[index + 3*stride]) * biases[2*n+1] / h;
    return b;

Here w and h are width and height of the network input, b.w and b.h are normalized width and height of the bonding box, x is last layer's output. It's not very clear what biases are though.

  • $\begingroup$ That's actually the same trick used in getting the standard deviation of a variational autoencoder. I don't get why they don't just use a normal relu though $\endgroup$ – Chester Cheng Jan 10 '18 at 13:36
  • $\begingroup$ @ChesterCheng yes relu will do the same but it won't get any gradient if the input is negative, actually other functions that maps a real number to a positive number would work as well for example $x^2$ and $log(1+exp(x))$. $\endgroup$ – dontloo Jan 10 '18 at 13:58
  • $\begingroup$ you are right! I guess they chose exp cus of the simple derivative $\endgroup$ – Chester Cheng Jan 10 '18 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.