3
$\begingroup$

Let's assume we have two matrices $A^{d\times 1}$ and $B^{1 \times e}$, and we define their product as $C^{d\times e}$. Assuming $A,B$ are real valued with all entries in $[-1,1]$.

I can intuitively see that the $L_2$ norm of $C$, where the norm is taken after flattening the matrix so we have one row and $d \times e$ columns, would be larger than the individual $L_2$ norms of $A$ and $B$ taken in the same way, because $C$ is much higher dimensional than $A$ and $B$. But, I was wondering if there is a formal result to support this and under what assumptions, i.e. the minimum size of $A$ and $B$ and any constraints on their scale etc.

$\endgroup$
3
  • $\begingroup$ There are at least two different notions of $L^2$ norm for matrices: the Frobenius norm $\|A\|=\sqrt{\sum_{i, j} \left|A_{i, j}\right|^2}$ and the operator norm $\|A\|=\sup_{\|x\|_2=1} \|A x\|_2$ (where $\|\cdot\|_2$ is the usual $L^2$ norm on $\mathbb{R}^n$). Which one do you mean here? $\endgroup$ Feb 12, 2019 at 17:10
  • $\begingroup$ This question seems to contain some inconsistencies which need clarification and edits. By "vector" would you mean matrix? By "dot product" would you mean matrix product? And in what sense is "$C$ ... higher dimensional than $A$"? If, for instance, $n=1,d=10^6,$ and $e=1,$ then $A$ has a million components and $C$ has just $1$ component! $\endgroup$
    – whuber
    Feb 12, 2019 at 17:16
  • 1
    $\begingroup$ @ArtemMavrin,@whuber, sorry, edited for clarity. $\endgroup$
    – lvdp
    Feb 12, 2019 at 17:26

1 Answer 1

2
$\begingroup$

For $p \ge 1,$ the $L_p$ norm of a vector $x=(x_1,\ldots, x_n)$ is defined to be

$$\|x\|_p = \left(\sum_{i=1}^n |x_i|^p\right)^{1/p}.$$

Notice that with $A^\prime=(a_i)$ and $B=(b_j),$ $C = (c_{ij}) = (a_i b_j).$ Thus

$$\|C\|_p^p = \sum_{ij} |c_{ij}|^p = \sum_{ij} |a_ib_j|^p = \sum_{ij} |a_i|^p |b_j|^p = \sum_i |a_i|^p \sum_j |b_j|^p = \|A\|_p^p \|B\|_p^p.$$

Taking $p^\text{th}$ roots shows the norms are equal. The case $p=2$ answers the question. There's no need to restrict the values of $a_i$ and $b_j$ and there's no restriction on the dimensions of the vectors, either. (The demonstration extends to all $p\ne 0,$ but for $p\lt 1$ this is no longer a norm.)

$\endgroup$
2
  • $\begingroup$ sorry, one more question, this only implies that the product of individual norms is equal to norm of the product. Can we say anything about the comparison of individual $||A||$ and individual $||B||$ with $||C||$? $\endgroup$
    – lvdp
    Feb 12, 2019 at 18:11
  • 1
    $\begingroup$ Yes, easily: the comparison of $\|A\|$ with $\|C\|$ is determined by $\|B\|$: if the latter is less than unity, then $C$ has smaller norm than $A$; otherwise, $C$ does not have smaller norm than $A.$ $\endgroup$
    – whuber
    Feb 12, 2019 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.