2
$\begingroup$

enter image description here

At the output of the final layer of yolo, a leaky-relu is applied to the output, so if we have negative values for the width and height, the cost function will return a null value since we would have square rooted a negative value at the second sum of the cost. Thus not able to update the weights using back prop.

Am I wrong about this or is there something I am missing here? If I am not wrong, how do we guarantee the width and height to be positive?

Improve this question
$\endgroup$
4
  • $\begingroup$ You are summing squared values; The sum will be at worst 0, assuming lambdas are positive. $\endgroup$
    – Vladislavs Dovgalecs
    Commented Jan 10, 2018 at 8:25
  • $\begingroup$ @VladislavsDovgalecs Sorry I don't see how that solves my problem, the square root would have still returned an imaginary value if the inside is negative, and if I do have an object in the cell, the term is not gonna become zero. $\endgroup$
    – Chester Cheng
    Commented Jan 10, 2018 at 8:33
  • $\begingroup$ You asked "how do we guarantee the output to be positive" and I understood it that way, how to prove the expression you wrote is positive. I apologize for having misunderstood your question. $\endgroup$
    – Vladislavs Dovgalecs
    Commented Jan 10, 2018 at 19:11
  • $\begingroup$ @VladislavsDovgalecs Thanks for your response, I will edit my question $\endgroup$
    – Chester Cheng
    Commented Jan 11, 2018 at 0:31

2 Answers 2

Reset to default
2
$\begingroup$

According to their source code, actually they use an exp operation to ensure $w$ and $h$ are non-negative values.

box get_region_box(float *x, float *biases, int n, int index, int i, int j, int w, int h, int stride)
{
    box b;
    b.x = (i + x[index + 0*stride]) / w;
    b.y = (j + x[index + 1*stride]) / h;
    b.w = exp(x[index + 2*stride]) * biases[2*n]   / w;
    b.h = exp(x[index + 3*stride]) * biases[2*n+1] / h;
    return b;
}

Here w and h are width and height of the network input, b.w and b.h are normalized width and height of the bonding box, x is last layer's output. It's not very clear what biases are though.

Improve this answer
$\endgroup$
3
  • $\begingroup$ That's actually the same trick used in getting the standard deviation of a variational autoencoder. I don't get why they don't just use a normal relu though $\endgroup$
    – Chester Cheng
    Commented Jan 10, 2018 at 13:36
  • $\begingroup$ @ChesterCheng yes relu will do the same but it won't get any gradient if the input is negative, actually other functions that maps a real number to a positive number would work as well for example $x^2$ and $log(1+exp(x))$. $\endgroup$
    – dontloo
    Commented Jan 10, 2018 at 13:58
  • $\begingroup$ you are right! I guess they chose exp cus of the simple derivative $\endgroup$
    – Chester Cheng
    Commented Jan 10, 2018 at 14:35
0
$\begingroup$

if we have negative values for the width and height

As in the original paper, the bounding box size is normalised between 0 and 1, hence will not be negative.

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.