I am a bit stuck implementing the Kernel Fisher Discriminant.
$$ J(\mathbf{w}) = \frac{\mathbf{w}^{\text{T}}\mathbf{S}_B^{\phi}\mathbf{w}}{\mathbf{w}^{\text{T}}\mathbf{S}_W^{\phi}\mathbf{w}} $$ $$ J(\mathbf{\alpha}) = \frac{\mathbf{\alpha}^{\text{T}}\mathbf{M}\mathbf{\alpha}}{\mathbf{\alpha}^{\text{T}}\mathbf{N}\mathbf{\alpha}}. $$
As far as I understand what I have to do is calculate the $\alpha$ which depends on $\mathbf{M}$ and $\mathbf{N}$
$$\mathbf{\alpha} = \mathbf{N}^{-1}(\mathbf{M}_2- \mathbf{M}_1).$$
In the case of $\mathbf{N}$, $\mathbf{N} = \displaystyle \sum_{j=1,2}\mathbf{K}_j(\mathbf{I}-\mathbf{1}_{l_j})\mathbf{K}_j^{\text{T}}$, with the $n^{\text{th}}, m^{\text{th}}$ component of $\mathbf{K}_j$ defined as $k(\mathbf{x}_n,\mathbf{x}_m^j)$ where $\mathbf{x_n}$ is all the observations and $\mathbf{x_m^j}$ is all the observations that belong to the class $j$
In the case of $\mathbf{M}_i$, $ (\mathbf{M}_i)_j = \frac{1}{l_i}\sum_{k=1}^{l_i}k(\mathbf{x}_j,\mathbf{x}_k^i).$ where $\mathbf{x_j}$ is all the observations and $\mathbf{x_k^i}$ is all the observations that belong to the class $i$.
Assuming that I understand that part, the big problem that I have is when I would like to use the $\alpha$ to assign a new observation in some class.
$$y(\mathbf{x}) = (\mathbf{w}\cdot\phi(\mathbf{x})) = \sum_{i=1}^l\alpha_ik(\mathbf{x}_i,\mathbf{x}).$$
However, when I try to implement this part, $\alpha$ has the dimensions of the total observations and as $x_i$ is a new observation the size of this vector is in the previous space and I really don't understand this part.
There are something that I am misunderstanding? How I have to use the $\alpha$ or $\mathbf{x}$?
What is the real meaning of $\mathbf{x}$ because I believe it has sometimes that has different meanings?