2
$\begingroup$

I'm reading the paper A Neural Attention Model for Sentence Summarization (Rush et al., EMNLP 2015) and came across the following:

Let the input consist of a sequence of $M$ words $\mathbf{x}_1, \dots , \mathbf{x}_M$ coming form a fixed vocabulary $\mathcal{V}$ of size $\vert \mathcal{V} \vert = V$. We will represent each word as an indicator vector $\mathbf{x}_i \in \{0, 1\}^V$ for $i \in \{1, \dots, M\}$, sentences as a sequence of indicators, and $\mathcal{X}$ as the set of possible inputs.

What does it mean to represent words as "indicator vectors?" The Wikipedia page for indicator vectors says that an indicator vector is basically a vector for a subset $T$ of set $S$ that is $0$ or $1$ depending on whether that element in $T$ is also in $S$.

Does the paper mean that each word is a one hot vector for the vocabulary? Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Exactly, each word is represented by a one hot vector of size $V$. In your quote, $\mathbf x_i\in \{0,1\}^V$ signifies this.

Improve this answer
$\endgroup$
1
  • $\begingroup$ That's also what I thought because nothing else makes sense. However, doesn't that notation mean that $\mathbf{x}_i$ is a binary vector rather than a one-hot vector? The notation is just what confused me. $\endgroup$
    – Sean
    Commented Jun 5, 2020 at 2:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.