Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I ran into this question which asks the identification of various terms for a linear regression function (f). I am confused about the "independent variable" definition.
What is the difference between a feature and an independent variable? Is "TV" in this case a feature as well as an independent variable?
"Feature" and "independent variable" are different terms for the same thing. "Feature" is more common in machine learning, whereas "independent variable" is more common in statistics. So yes, in this case, TV is both a feature and an independent variable.
Some more mostly equivalent terms are "covariate", "predictor", and "regression input".
Editing four years later to add: some psychologists and other social scientists reserve the term "independent variable" for a variable that has been randomly assigned or otherwise manipulated by the experimenter. "Predictor" or another word is then used for the other variables. But statisticians don't usually restrict the word like this.
It seems that the terms are used interchangeably, however, I found this explanation on a paper that used both:
We call “variable” the “raw” input variables and “features” variables constructed for the input variables. We use without distinction the terms “variable” and “feature” when there is no impact on the selection algorithms, e.g., when features resulting from a pre-processing of input variables are explicitly computed. The distinction is necessary in the case of kernel methods for which features are not explicitly computed
The usage mentioned here seems to be specific to the paper, so confirming that the two terms are synonyms.
Guyon & Elisseeff - An Introduction to Variable and Feature Selection - Journal of Machine Learning Research 3 (2003) 1157-1182
Take the example of a house - We want to build a model to calculate the price of the house based on certain criteria such as location, area (size), age, etc. Feature - all "input" to the model Variables - all Features + Price (the output).
In essence, Features (input variables) + output variable(s) of a model = Variables
Inputs can be considered as raw data like number of bathrooms,number of bedrooms whereas features are considered as some function of input like for example feature can be simply number of bathrooms or product of number of bathrooms and number of bedrooms
Features are different from variables to the extent that features methodologically precede variables. When algorithms are being used to identify potential variables within unstructured data, for example, the inputs (features:) cannot accurately be called variables.
