1
$\begingroup$

I'm reading the online book "Deep Learning" by Ian Goodfellow et al. . In section 9.5 Variants of the basic convolution function:

Directly quoting from the book,

"Suppose we want to train a convolutional network that incorporates strided convolution of kernel stack $\textsf{K}$ applied to multi-channel image $\textsf{V}$ with stride $s$ as defined by $c(\textsf{K},\textsf{V},s)$ as in Eq. 9.8. Suppose we want to minimize some loss function $J{(\textsf{V},\textsf{K})}$. During forward propagation, we will need to use $c$ itself to output $\textsf{Z}$, which is then propagated throuh the rest of the network and used to compute the cost function $J$. During back-propagation, we will receive a tensor $\textsf{G}$ such that $\textsf{G}_{i,j,k}=\frac{\partial J(\textsf{V},\textsf{K})}{\partial \textsf{Z}_{i,j,k}}$ "
\begin{equation}\tag{9.8}\textsf{Z}_{i,j,k}=c(\textsf{K}, \textsf{V}, s)_{i,j,k}=\sum_{l,m,n}\textsf{V}_{l,(j-1)\times s+m,(k-1)\times s+n}\textsf{K}_{i,l,m,n}\end{equation} Derivatives w.r.t. weights in the kernel: \begin{equation}\tag{9.11}g(\textsf{G},\textsf{V},s)_{i,j,k,l}=\frac{\partial J(\textsf{V},\textsf{K})}{\partial \textsf{K}_{i,j,k,l}}=\sum_{m,n}\textsf{G}_{i,m,n}\textsf{V}_{j,(m-1)\times s+k,(n-1)\times s+l}\end{equation} Gradient w.r.t. $\textsf{V}$: \begin{eqnarray}h(\textsf{K},\textsf{G},s)_{i,j,k}&=&\frac{\partial J(\textsf{V},\textsf{K})}{\partial\textsf{V}_{i,j,k}} \tag{9.12} \\ &=& \sum_{\substack{l,m\\ \textrm{s.t.}\\(l-1)\times s+m=j}} \sum_{\substack{n,p\\ \textrm{s.t.}\\(n-1)\times s+p=k}} \sum_q \textsf{K}_{q,i,m,p}\textsf{G}_{q,l,n} \tag{9.13} \end{eqnarray} Question: By the given tensor $\textsf{G}$ and equation 9.8, how can you derive 9.11-9.13? For eq. 9.11, I guess there should be a chain rule but I'm not sure why $\textsf{G}_{i,j,k}$ becomes $\textsf{G}_{i,m,n}$ summed over $m, n$...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.