enter image description here

At the output of the final layer of yolo, a leaky-relu is applied to the output, so if we have negative values for the width and height, the cost function will return a null value since we would have square rooted a negative value at the second sum of the cost. Thus not able to update the weights using back prop.

Am I wrong about this or is there something I am missing here? If I am not wrong, how do we guarantee the width and height to be positive?

  • $\begingroup$ You are summing squared values; The sum will be at worst 0, assuming lambdas are positive. $\endgroup$ Jan 10, 2018 at 8:25
  • $\begingroup$ @VladislavsDovgalecs Sorry I don't see how that solves my problem, the square root would have still returned an imaginary value if the inside is negative, and if I do have an object in the cell, the term is not gonna become zero. $\endgroup$ Jan 10, 2018 at 8:33
  • $\begingroup$ You asked "how do we guarantee the output to be positive" and I understood it that way, how to prove the expression you wrote is positive. I apologize for having misunderstood your question. $\endgroup$ Jan 10, 2018 at 19:11
  • $\begingroup$ @VladislavsDovgalecs Thanks for your response, I will edit my question $\endgroup$ Jan 11, 2018 at 0:31

1 Answer 1


According to their source code, actually they use an exp operation to ensure $w$ and $h$ are non-negative values.

box get_region_box(float *x, float *biases, int n, int index, int i, int j, int w, int h, int stride)
    box b;
    b.x = (i + x[index + 0*stride]) / w;
    b.y = (j + x[index + 1*stride]) / h;
    b.w = exp(x[index + 2*stride]) * biases[2*n]   / w;
    b.h = exp(x[index + 3*stride]) * biases[2*n+1] / h;
    return b;

Here w and h are width and height of the network input, b.w and b.h are normalized width and height of the bonding box, x is last layer's output. It's not very clear what biases are though.

  • $\begingroup$ That's actually the same trick used in getting the standard deviation of a variational autoencoder. I don't get why they don't just use a normal relu though $\endgroup$ Jan 10, 2018 at 13:36
  • $\begingroup$ @ChesterCheng yes relu will do the same but it won't get any gradient if the input is negative, actually other functions that maps a real number to a positive number would work as well for example $x^2$ and $log(1+exp(x))$. $\endgroup$
    – dontloo
    Jan 10, 2018 at 13:58
  • $\begingroup$ you are right! I guess they chose exp cus of the simple derivative $\endgroup$ Jan 10, 2018 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.