$\mathbf{SOLUTION 1:}$
Following Duda et al. (Pattern CLassification) which has an alternate solution to @lucas and in this case gives very easy to compute solution by hand. (Hope this alternate solution helps!! :))
In two class LDA the objective is:
$\frac{w^TS_Bw}{w^TS_Ww}$ which just means that increase the between class variance and decrease the within class variance.
where $S_B = (m_1-m_2)(m_1-m_2)^T$ and $S_W = S_1 + S_2$, here $S_1,S_2$ are covariance matrix and $m_1,m_2$ are means of class 1 and 2 respectively.
The solution of this generalized raleigh quotient is a generalized eigen value probem.
$S_Bw = \lambda S_Ww \rightarrow {S_W}^{-1}S_Bw = \lambda w $
The above formulation has a closed form solution. $S_B$ is a rank 1 matrix with basis $m_1-m_2$ so $w \propto {S_W}^{-1}(m1-m2)$ which can be normlizd to get the answer.
I just calculated the $w$ and got [0.5547;0.8321].
${S_W}^{-1}(m1-m2) = {(S_1 + S_2)}^{-1}(m1 - m2) = {(\begin{bmatrix}
2 & 0 \\
0 & 1
\end{bmatrix}
+ \begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix})}^{-1}(\begin{bmatrix}
0 \\
0
\end{bmatrix}
- \begin{bmatrix}
4 \\
4
\end{bmatrix}
)
={(\begin{bmatrix}
1/3 & 0 \\
0 & 1/2
\end{bmatrix})}(\begin{bmatrix}
0 \\
0
\end{bmatrix}
- \begin{bmatrix}
4 \\
4
\end{bmatrix}
)
= \begin{bmatrix}
-1.3333 \\
-2.0000
\end{bmatrix} \propto
\begin{bmatrix}
0.5547 \\
0.8321
\end{bmatrix} $
Ref: Pattern Classification by Duda, Hart, Stork
$\mathbf{SOLUTION 2:}$
Alternatively, it can be solved by finding eigen vector to the generalized eigen value problem.
$S_Bw = \lambda S_Ww$
A polynomial in lambda can be formed by $determinant(S_B - \lambda S_W)$ and the solutions to that polynomial will be the eigen value for $S_Bw = \lambda S_Ww$.
Now lets say you got a set of eigen values $\lambda_1,\lambda_2, ..., \lambda_n,$ as roots of the polynomial.
Now substitute $\lambda = \lambda_i, i \in \{1,2,..,n\}$ and get the corresponding eigen vector as solution to the linear system of equations $S_Bw_i = \lambda_i S_Ww_i$. By doing this for each i you can get a set of vectors $\{w_i\}_{i=1}^{n}$ and it is a set of eigen vectors as solutions.
$determinant(S_B - \lambda S_W) = \begin{bmatrix}
16 - 3\lambda & 16 \\
16 & 16 - 2\lambda
\end{bmatrix} =6\lambda^2 - 80\lambda$, So eigen values are roots to polynomial $6\lambda^2 - 80\lambda$.
So $\lambda= $ 0 and 40/3 are the two solutions.
For LDA, eigen vector corresponding to highest eigen value is the solution.
Solution to system of equation $(S_B - \lambda_i S_W)w_i = 0$ and $\lambda_i = 40/3$
which turns out to be $\begin{bmatrix}
16 - 3\lambda & 16 \\
16 & 16 - 2\lambda
\end{bmatrix}w_i \propto \begin{bmatrix}
-72 & 48 \\
48 & -32 \end{bmatrix}w_i = 0$
Solution to the above system of equation is $\begin{bmatrix}
-0.5547 \\
-0.8321 \end{bmatrix} \propto \begin{bmatrix}
0.5547 \\
0.8321 \end{bmatrix}$ which is same as previous solution.
Alternatively, we can say that $\begin{bmatrix}
0.5547 \\
0.8321 \end{bmatrix}$ lies in the null space of $\begin{bmatrix}
-72 & 48 \\
48 & -32 \end{bmatrix}$.
For two class LDA, eigen vector with highest eigen value is the solution. In general, for C class LDA, the first C - 1 eigen vectors to highest C - 1 eigen values constitute the solution.
This video explains how to compute eigen vectors for simple eigen value problem. ( https://www.khanacademy.org/math/linear-algebra/alternate_bases/eigen_everything/v/linear-algebra-finding-eigenvectors-and-eigenspaces-example )
Following is an example.
http://www.sosmath.com/matrix/eigen2/eigen2.html
Multi-class LDA:
http://en.wikipedia.org/wiki/Linear_discriminant_analysis#Multiclass_LDA
Calculating Null Space of a matrix:
https://www.khanacademy.org/math/linear-algebra/vectors_and_spaces/null_column_space/v/null-space-2-calculating-the-null-space-of-a-matrix
