Assume I have two neural networks, abstracted as two feature maps, parametrized by $\theta_x,\theta_y$ respectively. $\phi_x(x;\theta_x) \in \mathbb{R}^{h_1}$, $\phi_y(x;\theta_y) \in \mathbb{R}^{h_2}$ and we would like to perform canonical correlation analysis on the results of those feature maps, which means we want to maximize:

$$\max_{\theta_x, \theta_y, w_x, w_y} w_x^T C_{xy} w_y + \alpha [\text{min} (0, 1 - w_x^T C_{xx} w_x) + \text{min}(0, 1 - w_y^T C_{yy} w_y)] $$

where $C_{xy} = \mathbb{E}[\phi_x \phi_y^T]$, $C_{xx} = \mathbb{E}[\phi_x\phi_x^T]$, $C_{yy} = \mathbb{E}[\phi_y \phi_y^T]$

I now have to calculate the gradient of this function w.r.t. $\theta_x$ as a function of the Jacobian matrix $\frac{\partial \phi_x}{\partial \theta_x}$

Unfortunately, I don't really now to approach this. For example, what is the gradient of an expectation as a function of the Jacobian?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.