2
$\begingroup$

Bellow in this link http://cs231n.github.io/convolutional-networks/#case
it's written that the memory usage is just 24M
but don't we store the values of the parameters and activations in the forward pass?
So... now i think that the memory usage is the total number of parameters+ activations
the number is 24M +138M (without considering the number of biases)
Am i right?

$\endgroup$

2 Answers 2

2
$\begingroup$

I think that article is using "memory" as just counting the number of activations per image. Since you're right that of course we need to remember the parameters too, the total RAM used by the forward pass would be something like 93 MB per image in the batch, plus 4 bytes for each of the 138M parameters (about 552 MB). If you use a 6-image batch size, you'd be using approximately a gig of memory.

Actual usage will be somewhat more than that due to overheads in whatever implementation, and probably at least double that if you're keeping track of gradients to optimize things (running a backward pass).

$\endgroup$
2
  • $\begingroup$ thank you so much, but do you have any idea why this article is using "memory" as just counting the number of activations per image. $\endgroup$
    – floyd
    Commented Sep 18, 2017 at 3:35
  • 1
    $\begingroup$ It's the memory use per image. Not great terminology, but I don't think there's any deeper reason to it. $\endgroup$
    – Danica
    Commented Sep 18, 2017 at 3:35
0
$\begingroup$

For the detailed VGG memory footprint please take a look into: http://graphics.stanford.edu/courses/cs348v-18-winter/lectures/09_dnntrain.pdf, slide 22.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.