10
$\begingroup$

Random Fourier features provide approximations to kernel functions. They're used for various kernel methods, like SVMs and Gaussian processes.

Today, I tried using the TensorFlow implementation and I got negative values for half of my features. As I understand it, this shouldn't happen.

So I went back to the original paper, which---like I expected---says that the features should live in [0,1]. But its explanation (highlighted below) doesn't make sense to me: the cosine function can produce values anywhere in [-1,1] and most of the points displayed have negative cosine values.

I'm probably missing something obvious, but would appreciate it if someone can point out what it is.

Figure 1

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

Apparently, the highlighted sentence is wrong (or at least confusing): $z(x)$ can be negative. This isn't a problem because we only care about the inner product of $z$, not $z$ itself.

The "inner product" of $z$ only seemed incorrect when I used this method because I mixed up $z'z$ and $zz'$. Not because $z$ was wrong.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.