4
$\begingroup$

This is a question that I have in order to reconcile a difference in terminology.

In the linear regression setting, we have $y=\beta x + \theta$. Here, we call $x$ a variable. When we are trying to use this in a practical setting, suppose $x \in \mathbb{R}^n$. Then $x$ is represented as a feature vector. I think each feature could also be called a parameter. Is that true?

But if we can indeed call these parameters, then, since each $x_i$ is an iid variable, does this mean that these parameters are parameters of the distribution from which the $x_i$ are drawn? They can certainly be used to determine the parameters of the distribution.

Improve this question
$\endgroup$
1

4 Answers 4

Reset to default
5
$\begingroup$

Conventionally, "parameter" refers to the numbers that govern the statistical model or structure of data. Parameters are normally structural and descriptive. So in your case $\beta$ is a parameter, but $x$ is "data" and not a "parameter". If $x$ is drawn from a random distribution as in your comment, $\mu$ and $\sigma$ would also be parameters. "Feature" is machine learning lingo for a component of the data. In your case you just have one feature, since it's one-dimensional

Improve this answer
$\endgroup$
2
$\begingroup$

In statistics, $x$ is always called a variable, Beta and Theta are always parameters. In other fields, $x$ may be called a parameter. We can't help that but at least in statistics we are consistent (I hope).

Improve this answer
$\endgroup$
6
  • $\begingroup$ If $x$ is unidimensional, can't it be called a feature? Also, if $X \sim N(\mu, \sigma)$ wouldn't they then be called parameters of $X$? $\endgroup$
    – learning
    Commented Jan 13, 2017 at 2:50
  • $\begingroup$ So the "features" of each realisation $x_i$ is a "parameter" or can be used to determine the parameter of the distribution of $X$? Is that not right? I mean, can each "feature" of the variable $x$ also be called the "parameter" of the variable $x$? $\endgroup$
    – learning
    Commented Jan 13, 2017 at 2:57
  • $\begingroup$ The terminology for variables like x in classification are often called features. But about $X - N(m,s)$ is a random variable. We do not call it a parameter. $\endgroup$
    – Michael R. Chernick
    Commented Jan 13, 2017 at 3:05
  • 1
    $\begingroup$ If $ x$ is a random vector and $x_i$ is the ith component then $x_i$ is a univariate random variable. You are not going to get me to call it a parameter! In the field of machine learning I don't know what they call it but in statistics we are consistent. $\endgroup$
    – Michael R. Chernick
    Commented Jan 13, 2017 at 3:11
  • 3
    $\begingroup$ The statement is good for specific question and good for for Statistics 100 or Mathematical statistics 200 but otherwise too strong. In directional (circular, spherical statistics) trigonometric conventions trump statistical conventions and $\theta$ is utterly standard notation for an angle (direction), a variable, even for observed data. Still, many people spend entire statistical careers without ever needing to learn anything about directional statistics. $\endgroup$
    – Nick Cox
    Commented Jan 13, 2017 at 10:44
1
$\begingroup$

The way I've always looked at these semantics is the following: When collecting data a 'variable' is something you measure (in my field of biomedical research this could be sex, height, disease severity, diagnosis groups, etc.). In statistics, however, these variables are often 'dissected' into vectors of data which are ready for analysis. For linear regression for example, all your variables should be reclassified so that each parameter of your model represents one degree of freedom and receives its own coefficient. So in the case of continuous variables nothing changes (the coefficient represents the change in outcome when the continuous variable goes up or down one unit). For categorical variables with more than 2 levels however, you'll have to create dummy data vectors for all levels except the reference levels. This way, a four level categorical variable will be represented in your regression a three parameters (one for all levels except the reference), and each parameter/level will be assigned a coefficient during model fitting.

Compare these models to predict someone's weight

$x$ = length, in a one variable model and one parameter model to predict weight ($y$):

$y = a + Bx + e$ with

$x$ = US state in a one variable model, 50 parameter model to predict weight ($y$), with New York state as reference:

$y = a + B1x1 + ... + B50x50 + e$

where all states except NY are recoded as dummy parameters x1-x50 (including D.C. ofcourse)

In short, IMO in the context of regression a 'variable' constitutes the raw data, while a parameter reflects the actual $x$'s in your model.

Improve this answer
$\endgroup$
-2
$\begingroup$

Variable

In the field of mathematics, a variable defines as an element connected to a number, known as an estimation of the variable that is self-estimated, not completely determined, or ambiguous. The expression “variable” originates from the way that, when the argument (additionally called the “variable of the Function”) changes, then the estimate changes accordingly.

Parameter

A parameter, by and large, is an entity that can help in connecting or grouping a specific framework. That is, a parameter is a component of a system that is helpful, or basic. Inside and over different fields, watchful refinement must be kept up of the diverse utilizations of the term parameter and of different terms frequently connected with it. The reference of this site is https://researchpedia.info/difference-between-variable-and-parameter/

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I don't see how this helps or is in any sense an improvement on existing answers. The definition of parameter is hopelessly vague and could easily include any variable. It has no statistical flavour. The definition of variable assumes that numbers are involved, which is not correct. Result of coin toss is a variable and could be one of {heads, tails}, and so on. I agree with the idea underlying "watchful refinement must be kept up of the diverse utilizations of the term parameter" but what is needed here is focus on statistical uses (including machine learning, naturally). $\endgroup$
    – Nick Cox
    Commented Apr 17, 2020 at 12:02
  • $\begingroup$ it is mention from the point of computer science. All above answer are written from the point of maths. $\endgroup$
    – user66044
    Commented Apr 17, 2020 at 15:36
  • 1
    $\begingroup$ Computer science is not this vague and garbled. The reference is wholly mistaken. If this is a typical example of what is on researchpedia.info, then that is a site to be assiduously avoided. $\endgroup$
    – whuber ♦
    Commented Apr 21, 2020 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.