Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

Question

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert the x_i in this expression to big X (X is the whole dataset, a matrix)? The "alpha" in this expression is the Lagrange multiplier and the "xi" is the element of the dataset(a datapoint, it can be a vector or a single element). I tried using trace of matrices but it didn't take me very far.

Answer 1

Yes, it is possible. We have:

$$ w = \sum_{i=1}^n\alpha_ix_i=\alpha_1 x_1+ \alpha_2 x_2 + ... + \alpha_n x_n$$

Then, if we define: $$X = \begin{pmatrix} x_1^T\\ x_2^T\\ \vdots\\ x_n^T \end{pmatrix} \,\,\,\,\,\,\,\,\,\,\,\, \alpha=\begin{pmatrix} \alpha_1\\ \alpha_2\\ \vdots\\ \alpha_n \end{pmatrix}$$

Where each $x_i$ is a m-dimensional vector ($\text{size}(x_i)=m\times 1\rightarrow \text{size}(X)=n\times m)$). And each $\alpha_i$ is a scalar.

We can obtain $w$ by: $$ w =X^T\alpha$$

Where $w$ is a m-dimensional vector as well as each datapoint $x_i$.

---

Simple example:

Let's suppose we have the next two datapoints $x_1$ and $x_2$:

$$x_1=\begin{pmatrix} a\\ b\\ \end{pmatrix}\,\,\,\,\,\,\,\,\,\,\,\,\,\,x_2=\begin{pmatrix} c\\ d\\ \end{pmatrix}$$

Then, the data matrix is given by:

$$X=\begin{pmatrix} x_1^T\\ x_2^T\\ \end{pmatrix}=\begin{pmatrix} a & b\\ c & d\\ \end{pmatrix}$$

Further, suppose we have that $\alpha_1=e$ and $\alpha_2=f$, then: $$\alpha=\begin{pmatrix} \alpha_1\\ \alpha_2\\ \end{pmatrix}=\begin{pmatrix} e\\ f\\ \end{pmatrix}$$

Then, using the summatory: $$ w = \sum_{i=1}^n\alpha_ix_i= e\begin{pmatrix} a\\ b\\ \end{pmatrix}+f\begin{pmatrix} c\\ d\\ \end{pmatrix}$$

And if we use the matrix form:

$$ w = X^T\alpha=\begin{pmatrix} a & c\\ b & d\\ \end{pmatrix}\begin{pmatrix} e\\ f\\ \end{pmatrix}=e\begin{pmatrix} a\\ b\\ \end{pmatrix}+f\begin{pmatrix} c\\ d\\ \end{pmatrix}$$