26
$\begingroup$

Why is it most of the time (in many websites, articles or demonstration) the feature variable (columns) is denoted by a upper-case 'X' whereas the target variable is a lower-case 'y'?

Looks more like a coding standard to me. Ex.

X = df.iloc[:, :-1]
y = df.iloc[:, -1]

Just curious because I hardly ever use just a single letter to represent a variable storing meaningful data.

$\endgroup$
5
  • 8
    $\begingroup$ Consistency with linear algebra notation, I guess? The features usually form a matrix (typically denoted by uppercase) whereas the labels are usually 1d, forming a column vector (typically denoted by lowercase). $\endgroup$
    – galoosh33
    Commented Jan 27, 2019 at 15:49
  • 2
    $\begingroup$ "I hardly ever use just a single letter to represent a variable storing meaningful data." - the problem is that in common math typesetting, product signs are not written. Thus, it is unclear whether an expression $pi$ represents a single variable called "pi", or the product of two separate variables "p" and "i", $pi=p\times i$. To avoid this confusion, math-heavy disciplines very rarely use variables containing multiple letters. (When you implement an algorithm, yes, it is very good practice to replace single-letter variables by multi-letter ones, if only for easier search&replace.) $\endgroup$ Commented Jan 27, 2019 at 21:21
  • $\begingroup$ @StephanKolassa But in programming, a variable named pi does never mean a product of p and i (of course, unless you write pi = p * i or similarly). Even in the languages that allow juxtaposition for product they will be spaced out (p i, for example). (I personally think that allowing omission of the product symbol was of the strongest blunders in math notation history, since it restricts the variable name to one letter, so more or less meaningful designations are not possible without using indices; and the 1-letter names quickly run out, which forces us to use Greek, Gothic etc.) $\endgroup$
    – trolley813
    Commented Jan 28, 2019 at 7:58
  • 2
    $\begingroup$ @trolley813: the OP was explicitly asking about "websites , articles or demonstration", not programming - but then proceeded to noting (correctly) that this is not the convention in programming. There is simply a category confusion here, which I pointed out. $\endgroup$ Commented Jan 28, 2019 at 9:02
  • $\begingroup$ See also datascience.stackexchange.com/q/17598/55122, stackoverflow.com/q/52237376/10495893. $\endgroup$ Commented Jun 18, 2021 at 18:12

3 Answers 3

30
$\begingroup$

The question about why $X$ and $y$ are popular choices in mathematical notions has been answered in the History of Science and Mathematics SE website: Why are X and Y commonly used as mathematical placeholders? (In short: cause Descartes said so!)

In terms of Linear Algebra, it is extremely common to use capital Latin letters for matrices (e.g. design matrix $X$) and lowercase Latin letters for vectors (response vector $y$). Standard textbooks on the use of matrices in Statistics (e.g. Matrix Algebra Useful for Statistics by Searle, Matrix Algebra From a Statistician's Perspective by Harville and Matrix Algebra: Theory, Computations, and Applications in Statistics by Gentle) utilise this convention too, so it has become a standard way to denote things.

$\endgroup$
7
$\begingroup$

Before you collect any data values on the feature and target variables, these variables can be considered to be random variables provided a random mechanism will be used to select the subjects who will generate these values. In that case, the correct notation for these variables is Y and X (i.e., upper case letters for both).

Recall that the value of a random variable is unknown prior to collecting the data, though its behaviour in the long run can be predicted using probability laws. However, once we collect the data, that value becomes known.

After you collect all desired data values on the feature and target variables, you can use the lower case notation to denote the collection of data values corresponding to the target variable (y) and the feature variables (x). If you have a single feature variable, x is a vector of data values. If you have multiple feature variables, x is a matrix of data values, having one column per feature variable. Usually, y is a vector of data values.

So the upper case notation refers to "random (hence unknown)", while the lower case notation refers to "known". Alternatively, the upper case notation refers to "before collecting the data", while the lower case notation refers to "after collecting the data".

Sadly, the literature is not at all consistent in the use of this notation, which is why you see the (y,X) notation you mention in your question.

$\endgroup$
2
$\begingroup$

To understand when to use lowercase or uppercase, we need to know what is represented in X_train or X_test. It is a capital letter X to represent a 2-D matrix. And for y_train and y_test, it is a small letter y to represent a 1-D vector.

Mathematically, it is a common notation for Linear Algebra to use uppercase Latin letters for matrices (e.g. matrix X) and lowercase Latin letters for vectors (vector y).

In data science, the feature matrix X is a collection of many columns of feature values. For example a df with 1 target, 20 features and 1000 data records will have the shape of shape (1000, 21). So we will define the feature matrix X to have the shape (1000, 20). Whereas the target label y is a column of values having the shape (1000, 1).

After applying train_test_split() on X and y with test_size=0.25, I would expect:
X_train to be a 2-D matrix (750, 20)
y_train to be a 1-D vector (750, 1)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.