# What is "feature space"?

What is the definition of "feature space"?

I understand what's going on, especially for CART, but I think that there is some definition that I have missed.

Is there a general definition of "feature space"?

Is there a definition that will give me more insight into SVM kernels and/or CART?

• Feature space just refers to the collections of features that are used to characterize your data. For example, if your data is about people, your feature space might be (Gender, Height, Weight, Age). In a SVM, we might want to consider a different set of characteristics to describe the data, such as (Gender, Height, Weight, Age^2, Height / Weight) etc; this is the mapping to another feature space Dec 22 '12 at 17:14
• Would you please give the names/titles of you reading? Aug 7 '19 at 12:40

## Feature Space

Feature space refers to the $n$-dimensions where your variables live (not including a target variable, if it is present). The term is used often in ML literature because a task in ML is feature extraction, hence we view all variables as features. For example, consider the data set with:

Target

1. $Y \equiv$ Thickness of car tires after some testing period

Variables

1. $X_1 \equiv$ distance travelled in test
2. $X_2 \equiv$ time duration of test
3. $X_3 \equiv$ amount of chemical $C$ in tires

The feature space is $\mathbf{R}^3$, or more accurately, the positive quadrant in $\mathbf{R}^3$ as all the $X$ variables can only be positive quantities. Domain knowledge about tires might suggest that the speed the vehicle was moving at is important, hence we generate another variable, $X_4$ (this is the feature extraction part):

• $X_4 =\frac{X_1}{X_2} \equiv$ the speed of the vehicle during testing.

This extends our old feature space into a new one, the positive part of $\mathbf{R}^4$.

## Mappings

Furthermore, a mapping in our example is a function, $\phi$, from $\mathbf{R}^3$ to $\mathbf{R}^4$:

$$\phi(x_1,x_2,x_3) = (x_1, x_2, x_3, \frac{x_1}{x_2} )$$

• How does this differ from a sample space in probability theory? Just asking. I would like to know. Dec 23 '12 at 2:43
• It's is very similar, if not identical. If you consider the data-generating distribution $D$, then the feature-space is identical to the support of $D$. Dec 23 '12 at 4:25
• I would say that, as Pilon's example shows, feature space can be increased by extracting some new features. Sample space in probability can't. It's exhaustive, feature spaces aren't. Apr 5 '16 at 12:56
• @Cam.Davidson.Pilon someone wsa inspired by your answer it seems: dataorigami.net/blogs/napkin-folding/… Aug 6 '19 at 15:22
• @AIM_BLB that's me! Aug 6 '19 at 15:44