2
$\begingroup$

Spark doc contains this:

An Estimator abstracts the concept of a learning algorithm or any algorithm that fits or trains on data. Technically, an Estimator implements a method fit(), which accepts a DataFrame and produces a Model, which is a Transformer. For example, a learning algorithm such as LogisticRegression is an Estimator, and calling fit() trains a LogisticRegressionModel, which is a Model and hence a Transformer.

But Wikipedia says:

An "estimator" or "point estimate" is a statistic (that is, a function of the data) that is used to infer the value of an unknown parameter in a statistical model. The parameter being estimated is sometimes called the estimand. It can be either finite-dimensional (in parametric and semi-parametric models), or infinite-dimensional (semi-parametric and non-parametric models). If the parameter is denoted $\theta$ then the estimator is traditionally written by adding a circumflex over the symbol: $\widehat {\theta }$. Being a function of the data, the estimator is itself a random variable; a particular realization of this random variable is called the "estimate". Sometimes the words "estimator" and "estimate" are used interchangeably.

I would have called the machine learning model itself the "estimator". Are the 2 definitions above close enough, or off, or are there important distinctions to be aware of?

$\endgroup$
  • $\begingroup$ This is part of the problem between CS/ML people and stats. There's a difference in language. What they mean by Estimator is more of a method/implementation not a point estimate. $\endgroup$ – Jon Jan 18 '17 at 22:15
  • 1
    $\begingroup$ @Jon you are confusing estimator (a function) with estimate (it's outcome given the sample), see stats.stackexchange.com/questions/7581/… $\endgroup$ – Tim Jan 18 '17 at 22:24
  • $\begingroup$ What else would you call something that estimates the parameters of the model? $\endgroup$ – david25272 Jan 19 '17 at 2:30
  • $\begingroup$ @david25272 - a procedure, algorithm, method, recipe... for the part that does the calculation of the parameters of the model? $\endgroup$ – Frank Jan 19 '17 at 3:48
  • $\begingroup$ I've always thought of algorithms as specific (e.g. OLS, Iteratively Reweighted LS) examples of estimators. $\endgroup$ – david25272 Jan 19 '17 at 4:58
7
$\begingroup$

Spark's documentation refers to internal names of Spark's procedures and objects, not to theoretical concepts. In statistics estimator is basically a procedure that estimates something given the data and returns an estimate.

It is hard to answer your question without knowing what you mean by model. Statistical model describes the phenomenon of interest in terms of probability theory (notice that there are also other models, e.g. mechanistic models). It is not yet an estimator of anything, but just an abstract description of the problem. You need an estimator to estimate the parameters of your model (also in case of so-called non-parametric models). In machine learning people often call algorithms used to estimate something as models, but in general: model is a theoretical description of the problem and estimator is a procedure that generates the estimates of its parameters. Estimator is a general procedure, not its implementation, so function foo() in software XYZ is not an estimator, but rather it's an implementation of some estimator.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ (+1) My understanding was that statistical models - which have a stochastic & deterministic component - comprise mechanistic models & empirical models. The form of the deterministic component of the former is decided by theory; that of the latter by how much flexibility it seems prudent to allow - representing predictors with a polynomial, spline, fourier term, &c. basis. $\endgroup$ – Scortchi - Reinstate Monica Jan 19 '17 at 1:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.