1
$\begingroup$

In Google developers site, the meaning of the word sparse seems to contradict between the following two definitions:

  1. sparse feature:

Feature vector whose values are predominately zero or empty...

  1. sparse representation:

A representation of a tensor that only stores nonzero elements...

Can someone please help me make better sense out of this?

$\endgroup$
2
  • 1
    $\begingroup$ There's no contradiction. If you only store the non-zero elements of an object that is mostly zeros, you need less storage. That's all. $\endgroup$
    – Sycorax
    Jun 15, 2022 at 17:45
  • $\begingroup$ @Sycorax, the dense representation example in the second definition has mostly zeros and sparse has none. Doesn't this contradict with the 1st definition? $\endgroup$ Jun 15, 2022 at 18:35

2 Answers 2

1
$\begingroup$

These concepts reinforce one another once you understand the visual purpose of the latter. A sparse object is a vector, matrix, etc., that contains a lot of zeroes in it. Consequently, a more useful representation for showing that thing is the representative form that omits the zeroes, which allows us to concentrate on the relatively small number of non-zero values. The "sparse feature" and "sparse representation" shown here is an example of this.

I get why you say that these terms appear to be contradictory --- the name "sparse representation" for the latter is really the opposite of what it actually is. The so-called "sparse representation" is actually more like a "sparse-part-omitted representation" of the original thing. Presumably the originators of these concepts felt that literal description was a bit too much of a mouthful, and they probably figured that the names given would make sense in the context of understanding why it is good to represent sparse objects in this reduced way.

$\endgroup$
1
$\begingroup$

In a sparse array, the thing that is sparse is the storage: only nonzero elements are stored.

The two arrays in OP's question have the exact same number of 0s. The purpose of the image in the question is to illustrate that the dense array stores more than 900,000 entries, with nearly all of them 0, while the sparse array only stores the 3 non-zero entries. The absent entries are implicitly known to be 0 because their cells aren't stored in the sparse array.

A sparse array isn't much different than making a grocery list: you could either write down a sparse representation:

  • butter
  • eggs
  • milk

or you could write down a dense representation:

  • butter: 1
  • cereal: 0
  • eggs: 1
  • lime: 0
  • mango: 0
  • soup: 0
  • steak: 0
  • ...and so on, for every other item in the grocery store.

The "sparse grocery list" is going to be easier to read and understand. Likewise, a sparse array (tensor, matrix, vector) can be more economical to work with (less memory, some math operations are more efficient) than a dense array.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.