I am trying to understand the Matrix and Vector Notations on page 2 here: (the page is also pasted below, to make it easier to explain the problem).

Problem: For equation (2), I think it should be $\textbf{h}^{T}$ instead of $\textbf{h}$ for matrix multiplication to make sense:

enter image description here

Why I think it should be $\textbf{h}^{T}$

$\textbf{x}$ is a Vx1 dimensional vector. $\textbf{W}$ is a VxN dimensional Matrix. Below I re-write equation (1) with dimensions for each vector and matrix:

$$ \textbf{h}_{1xN} = \textbf{x}^{T}_{1xV}\textbf{W}_{VxN} = {\textbf{v}_{w_{I}}}_{1xN}$$

$\textbf{v}^{'}_{w_{j}}$ is a Nx1 dimensional j-th column of matrix $\textbf{W}^{'}$ and $u_{j}$ is a scalar. Therefore rewriting equation (2) with dimensions (you can only multiple when you use $\textbf{h}^{T}$ and not $\textbf{h}$ as shown in figure):

$${u_{j}}_{1x1} = {\textbf{v}^{'}_{w_{j}}}^{T}_{1xN}.\textbf{h}^{T}_{Nx1}$$


1 Answer 1


${v'_{w_j}}^T$ and $h$ are both row vectors, as you mention. I think they're using the center dot between them to denote the dot product, which seems ok. If it were matrix multiplication notation, then I think you're right that it would be ${v'_{w_j}}^T h^T$.

  • $\begingroup$ Wow !! yes. You are correct. I never realized that "dot product" is different than matrix multiplication. If you are using "dot product" you can say $\textbf{x}\cdot\textbf{y}$ where both $\textbf{x}$ and $\textbf{y}$ are 1xN dimensional vectors: math.odu.edu/~bogacki/math316/transp/1_3 $\endgroup$ May 31, 2016 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.