What is the definition of "feature space"?

For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space".

I understand what's going on, especially for CART, but I think that there is some definition that I have missed.

Is there a general definition of "feature space"?

Is there a definition that will give me more insight into SVM kernels and/or CART?

  • 8
    $\begingroup$ Feature space just refers to the collections of features that are used to characterize your data. For example, if your data is about people, your feature space might be (Gender, Height, Weight, Age). In a SVM, we might want to consider a different set of characteristics to describe the data, such as (Gender, Height, Weight, Age^2, Height / Weight) etc; this is the mapping to another feature space $\endgroup$
    – JCWong
    Dec 22, 2012 at 17:14
  • $\begingroup$ Would you please give the names/titles of you reading? $\endgroup$
    – JJJohn
    Aug 7, 2019 at 12:40

1 Answer 1


Feature Space

Feature space refers to the $n$-dimensions where your variables live (not including a target variable, if it is present). The term is used often in ML literature because a task in ML is feature extraction, hence we view all variables as features. For example, consider the data set with:


  1. $Y \equiv$ Thickness of car tires after some testing period


  1. $X_1 \equiv$ distance travelled in test
  2. $X_2 \equiv$ time duration of test
  3. $X_3 \equiv$ amount of chemical $C$ in tires

The feature space is $\mathbf{R}^3$, or more accurately, the positive quadrant in $\mathbf{R}^3$ as all the $X$ variables can only be positive quantities. Domain knowledge about tires might suggest that the speed the vehicle was moving at is important, hence we generate another variable, $X_4$ (this is the feature extraction part):

  • $X_4 =\frac{X_1}{X_2} \equiv$ the speed of the vehicle during testing.

This extends our old feature space into a new one, the positive part of $\mathbf{R}^4$.


Furthermore, a mapping in our example is a function, $\phi$, from $\mathbf{R}^3$ to $\mathbf{R}^4$:

$$\phi(x_1,x_2,x_3) = (x_1, x_2, x_3, \frac{x_1}{x_2} )$$

  • $\begingroup$ How does this differ from a sample space in probability theory? Just asking. I would like to know. $\endgroup$
    – Placidia
    Dec 23, 2012 at 2:43
  • $\begingroup$ It's is very similar, if not identical. If you consider the data-generating distribution $D$, then the feature-space is identical to the support of $D$. $\endgroup$ Dec 23, 2012 at 4:25
  • 1
    $\begingroup$ I would say that, as Pilon's example shows, feature space can be increased by extracting some new features. Sample space in probability can't. It's exhaustive, feature spaces aren't. $\endgroup$ Apr 5, 2016 at 12:56
  • $\begingroup$ @Cam.Davidson.Pilon someone wsa inspired by your answer it seems: dataorigami.net/blogs/napkin-folding/… $\endgroup$
    – ABIM
    Aug 6, 2019 at 15:22
  • 1
    $\begingroup$ @AIM_BLB that's me! $\endgroup$ Aug 6, 2019 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.