Why use sinusoidal along embedding dimension in positional encoding in transformers?

Question

Why do we need sinusoidal function along the embedding dimension in positional encoding in transformers? Shouldn't sinusoidal function along time dimension be enough?

This question is derived from this answer as I cannot comment on it yet.

As mentioned in the linked answer, suppose a word is embedded with a vector: $e_1, e_2, ..., e_d$. If the same word occurs at two different positions, we would like to encode that information using the positional encoding.

I understand adding sinusoidal function along the position/time dimension but what is the rationale behind varying the positional encoding along $e_1, e_2, ..., e_d$. Say the positional encoding for two positions is $p_1, p_2, ..., p_d$ and $p^{'}_1, p^{'}_2, ..., p^{'}_d$. Why can't $p_1 = p_3 ... = \sin(f(t))$ and $p_2 = p_4 ... = \cos(f(t))$ where $f(t)$ is a function of time step?

and similarly for $p^{'}$

Here is a related question

Answer 1

Here is what you may need to read:

Where do we get these positional embeddings? A simple and effective approach is to start with randomly initialized embeddings corresponding to each possible input position up to some maximum length. For example, just as we have an embedding for the word fish, we’ll have an embedding for the position 3. As with word embeddings, these positional embeddings are learned along with other parameters during training. To produce an input embedding that captures positional information, we just add the word embedding for each input to its corresponding positional embedding. This new embedding serves as the input for further processing.

A potential problem with this approach is that there will be plenty of training examples for the initial positions in our inputs and correspondingly fewer at the outer length limits. These latter embeddings may be poorly trained and may not generalize well during testing. An alternative approach to positional embeddings is to choose a static function that maps an integer inputs to real-valued vectors in a way that captures the inherent relationships among the positions. That is, it captures the fact that position 4 in an input is more closely related to position 5 than it is to position 17. A combination of sine and cosine functions with differing frequencies was used in the original Transformer work.

Source: Speech and Language Processing.

Answer 2

If you apply the same positional embedding (PE) to all dimensions of a vector, say e_1, the PE is 1-d after all. It's natural that the differences between 100 10-d random PE are bigger than those in 100 1-d random PE. In other words, you are providing much more positional information using different PE for different dimensions than just using the same.