How can I calculate cosine distance with multiple feature vectors and weigh them? I have a dataset of text documents and I'm calculating pairwise cosine distances among them. For each document I have a bag of words vector, a vector built from entities extracted from the document, and the time the document was created. Each type of vector is of different dimensions.
I want the distance calculation to use all of these features, but such that the distance is biased towards the concepts vector. I have been calculating the distances for each type of vector independently then combining them as a weighted average, but I suspect there's a more correct approach.
 A: Just do it.
Cosine boils down to computing scalar products (with each other, and each vector with itself when computing the magnitude),
$$\langle a,b\rangle = \sum_i a_ib_i$$
which can trivially be weighted
$$\langle a,b\rangle_\Omega = \sum_i \omega_ia_ib_i$$
Choose $\omega_i$ such that each feature set has the same sum of weights. For example, choose $\omega_i=1/d$ where $d$ is the number of features in this features space.
You can move the $\omega_i$ term out if it is constant for all attributes. You will likely end up with something as easy as this:
$$\langle a,b\rangle = \frac{1}{|F_1|}\langle F_1(a), F_1(b)\rangle + \frac{1}{|F_2|}\langle F_2(a), F_2(b)\rangle + \ldots$$
which is the same as scaling/weighting each attribute in each of the feature spaces by $1/|F_i|$ where $|F_i|$ is the dimensionality of the features space.
You can further simplify the process if you first normalize all vectors to unit length (in each feature space separately). Then $||a||=||b||=1$, and cosine does become the scalar product:
$$\cos = \frac{\langle a,b\rangle}{||a||\cdot ||b||} = \langle \frac{a}{||a||},\frac{b}{||b||}\rangle =_{\text{if }||a||=||b||=1} \langle a,b\rangle$$
As this equation shows (using bilinearity of the scalar product), you do get the same result.
Let $F_i^\prime(a)=F_i(a)/||F_i(a)||$ be the data normalized to $L_2$ norm 1. Then you can just choose
$$\text{sim}(a,b) = \frac{1}{k}\sum_{i=1}^k \langle F_i^\prime(a),F_i^\prime(b)\rangle$$
If $F^\prime(a) = \frac{1}{k} \mathop{concat}_{i=1}^k F_i^\prime(a)$ is the concatenation of the L2 unit normalized vectors; divided by $k$, then this is the classic scalar product/cosine:
$$\text{sim}(a,b) = \langle F^\prime(a), F^\prime(b)\rangle$$
In other words, normalize data in each feature space independently to unit length, then each features space gets the same weight.
