3
$\begingroup$

The term “vector” is used heavily in both math and machine learning. In math/physics it’s a geometric object that has both magnitude and direction, while in machine learning it’s a data structure. In both cases they are essentially just a list of numbers that can be represented as pointing in vector space.

In machine learning the vectors we speak of are “feature vectors” since the list of numbers we work with is a row in a dataset, and each row is a list of numbers whose elements are the values for each feature (column in the dataset). So in this case a “vector” is defined as a row in the dataset.

enter image description here

enter image description here

But there are other situations where it seems to make more sense to consider a column as a vector. For example, if we wanted to know the correlation between 2 variables in a dataset we would be looking to compare 2 columns. Correlation can be interpreted geometrically using vectors in vector space. This means a column would be considered a vector.

enter image description here

But how can the list of numbers in a column be a vector in vector space if each element in the list (each row value for that column) belongs to only one feature? In other words, if we were visualizing the vector in vector space, how do all the numbers in the list that make the vector get plotted? In the case of a feature vector this is obvious, since each column is a dimension in the plot, but in the case where a single column is itself a vector it isn’t obvious how this would be plotted in vector space.

enter image description here

$\endgroup$
1
  • 2
    $\begingroup$ The each column is a sample vector for one feature. It would be plotted in sample space, not feature space. Features space vs. Sample space is entirely analogous to the usual linear algebra ideas of the Column space vs. Row space of a matrix. $\endgroup$ Nov 7, 2020 at 15:05

2 Answers 2

5
$\begingroup$

The same information can be represented by both column and row vector. For example, consider the following vector:

$$\mathbf{x} = [ x_1, x_2, x_3]$$

this vector can be transposed as:

$$\mathbf{x}^T = \begin{bmatrix}x_1 \\x_2 \\ x_3\end{bmatrix}$$

Both of the above vectors contain the same information. One is just a transpose of the another.

$$[ x_1, x_2, x_3]^T = \begin{bmatrix}x_1 \\x_2 \\ x_3\end{bmatrix}$$

The information is not changed with this transformation. If $x_1$ is $x$ coordinate in $\mathbb{R}^3$ space, $x_2$ is $y$ coordinate and $x_3$ is $z$ coordinate they still remain such after taking the transpose.

Edit after the edit I am bit confused about the question. If you have an array of numbers like:

$$ \begin{bmatrix}x_{11} & x_{12} & x_{13} \\x_{21} & x_{22} & x_{23} \\ x_{31} & x_{32} & x_{33} \end{bmatrix}$$

where rows could represent observations and columns represent different variables (e.g. column 1 could be age and row 1 person A making $x_{11}$ an age of person A).

In such situation you could subset the data from the array into both row and column vectors. A column vector $[x_{11}, x_{12} , x_{13}]$ which would be vector of age and other characteristics, as well as you can have $ \begin{bmatrix}x_{11} \\x_{21} \\ x_{31}\end{bmatrix}$ column vector of in this case age for different observations.

However, for some subsequent mathematical operations you can still transpose these vectors. So for example, the 'age' vector $ \begin{bmatrix}x_{11} \\x_{21} \\ x_{31}\end{bmatrix} = \mathbf{a}$ which contains all ages across all observations would still contain the same information if you would apply transpose making it a new column vector that is:

$$ \mathbf{a}^T = [x_{11},x_{21},x_{31}].$$

$\endgroup$
5
  • $\begingroup$ But I am talking about taking different values altogether, so they cannot possibly represent the same information. The values in a feature vector are entirely different than the values in the column of a dataset. See figures added to question. $\endgroup$
    – Cybernetic
    Nov 7, 2020 at 15:21
  • $\begingroup$ @Cybernetic does the edit help? $\endgroup$
    – 1muflon1
    Nov 7, 2020 at 15:32
  • $\begingroup$ Right, so I am talking about your last example, “age for different observations.” If I understand, this last example can just as easily be plotted in vector space since this list of numbers would be a point in high-dimensional space, where we have a dimension in the plot for each value in the list. This is much more abstract since it isn’t obvious what the dimension is in this example, whereas for a feature vector the dimension is an actual characteristic in the data. But in abstract terms both a row (feature) vector and column vector can be represented in vector space. Correct? $\endgroup$
    – Cybernetic
    Nov 7, 2020 at 15:38
  • 1
    $\begingroup$ @Cybernetic yes although, I would not say it’s not clear what dimensions are, the dimensions would simply be age1, age2 and age3 - sure that is probably not how dimensions are usually introduced and it is a bit more abstract but I would not say that these are not clear dimensions. $\endgroup$
    – 1muflon1
    Nov 7, 2020 at 15:51
  • $\begingroup$ Excellent, thank you. $\endgroup$
    – Cybernetic
    Nov 7, 2020 at 15:52
0
$\begingroup$

Your first drawing is a feature space, the second drawing should be a record space. Where each vector represents the similarity of one column to another. The same vectors mean the same columns numerically.

$\endgroup$
1
  • 2
    $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Feb 28, 2023 at 7:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.