While formulating a certain algorithm, I have to describe the following operation between vectors.

Given $x, y \in \mathbb{R^d}$, the function $f(\cdot, \cdot): \mathbb{R^d} \times \mathbb{R^d} \to \mathbb{R^{d^2}}$ is defined as

$$f(x,y) = [x_1 y_1, x_1 y_2, \cdots, x_1 y_d, x_2 y_1, x_2 y_2, \cdots, x_d y_d]$$

As you can see this function is fairly similar to Cartesian product from set theory, which makes me think this might be a well-known operation with an specific name. Am I right?

I am aware that this operation can be achieved by allying matrix multiplication between a column-vector and a row-vector (i.e. $x^Ty$), but my operation needs to be generalizable to have more than two input vectors, e.g. $f(\cdot, \cdot, \cdot): \mathbb{R^d} \times \mathbb{R^d}\times\mathbb{R^d} \to \mathbb{R^{d^3}}$.

  • 3
    $\begingroup$ Lookup outer product or tensor product. $\endgroup$ – Matthew Gunn Feb 6 '17 at 1:00
  • $\begingroup$ @Lope I notice there are two terms at the end of the bracket $x_d$, $y_d$. Do you really mean to include those? In that case, it's not quite the standard outer/kronecker product suggested. $\endgroup$ – horaceT Feb 7 '17 at 1:08
  • $\begingroup$ Those are not two terms, is the single term $x_d\cdot y_d$. $\endgroup$ – Daniel López Mar 30 '17 at 12:27

Your direction question:

  • In the two-dimensional case, you're looking for the outer product. The Euclidean inner product is $x^T y$ while the outer product is $xy^T$. The outer product generates a matrix.

  • In the general case, you're asking for a tensor product.

  • What you're asking for is also the Kronecker product of the vectors. (See Rodrigo de Azevedo's answer.)

Some programming comments:

  • Be aware of row major or column major, that is, how to handle multi-dimensional arrays.
  • This is standard linear algebra, and I would recommend using linear algebra libraries if you're doing any medium to heavy lifting or if performance matters. Don't reinvent the wheel. Some discussion is here.
    • BLAS/LAPACK are standard, stable, and fast. But calling BLAS/LAPACK functions is a pain.
  • $\begingroup$ Related? It is the Kronecker product. $\endgroup$ – Rodrigo de Azevedo Feb 7 '17 at 0:47
  • $\begingroup$ @RodrigodeAzevedo Point taken. The qualifier "related" shouldn't have been there. $\endgroup$ – Matthew Gunn Feb 7 '17 at 1:18

Let $\mathrm x, \mathrm y \in \mathbb R^d$. The Kronecker product of these two $d$-dimensional column vectors is the following $d^2$-dimensional column vector

$$\mathrm x \otimes \mathrm y = \begin{bmatrix} x_1 \mathrm y\\ x_2 \mathrm y\\ \vdots\\ x_d \mathrm y\\\end{bmatrix} = \begin{bmatrix} x_1 y_1\\ x_1 y_2\\ \vdots\\ x_1 y_d\\ x_2 y_1\\ x_2 y_2\\ \vdots\\ x_2 y_d\\ \vdots\\ x_d y_1\\ x_d y_2\\ \vdots\\ x_d y_d\end{bmatrix}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.