Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

What is the definition of "feature space"?

For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space".

I understand what's going on, especially for CART, but I think that there is some definition that I have missed.

Is there a general definition of "feature space"?

Is there a definition that will give me more insight into SVM kernels and/or CART?

share|improve this question
1  
Feature space just refers to the collections of features that are used to characterize your data. For example, if your data is about people, your feature space might be (Gender, Height, Weight, Age). In a SVM, we might want to consider a different set of characteristics to describe the data, such as (Gender, Height, Weight, Age^2, Height / Weight) etc; this is the mapping to another feature space –  JCWong Dec 22 '12 at 17:14
add comment

1 Answer

up vote 6 down vote accepted

Feature Space

Feature space refers to the $n$-dimensions where your variables live (not including a target variable, if it is present). The term is used often in ML literature because a task in ML is feature extraction, hence we view all variables as features. For example, consider the data set with:

Target

  1. $Y \equiv$ Thickness of car tires after some testing period

Variables

  1. $X_1 \equiv$ distance travelled in test
  2. $X_2 \equiv$ time duration of test
  3. $X_3 \equiv$ amount of chemical $C$ in tires

The feature space is $\mathbf{R}^3$, or more accurately, the positive quadrant in $\mathbf{R}^3$ as all the $X$ variables can only be positive quantities. Domain knowledge about tires might suggest that the speed the vehicle was moving at is important, hence we generate another variable, $X_4$ (this is the feature extraction part):

  • $X_4 =\frac{X_1}{X_2} \equiv$ the speed of the vehicle during testing.

This extends our old feature space into a new one, the positive part of $\mathbf{R}^4$.

Mappings

Furthermore, a mapping in our example is a function, $\phi$, from $\mathbf{R}^3$ to $\mathbf{R}^4$:

$$\phi(x_1,x_2,x_3) = (x_1, x_2, x_3, \frac{x_1}{x_2} )$$

share|improve this answer
    
How does this differ from a sample space in probability theory? Just asking. I would like to know. –  Placidia Dec 23 '12 at 2:43
    
It's is very similar, if not identical. If you consider the data-generating distribution $D$, then the feature-space is identical to the support of $D$. –  Cam.Davidson.Pilon Dec 23 '12 at 4:25
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.