19
$\begingroup$

Is there a conceptual or procedural difference between fitting a model to data and fitting data to model? An example of the first wording can be seen in https://courses.washington.edu/matlab1/ModelFitting.html, and of the second in https://reference.wolfram.com/applications/eda/FittingDataToLinearModelsByLeast-SquaresTechniques.html.

$\endgroup$
  • 7
    $\begingroup$ +1 I am not impressed by the second link, but I am entertained. $\endgroup$ – The Laconic Mar 24 at 2:38
  • $\begingroup$ Many models fits current data, but data typically fits best one model $\endgroup$ – Agnius Vasiliauskas Mar 25 at 9:45
34
$\begingroup$

Pretty much every source or person I've ever interacted with except the Wolfram source you linked refers to the process as fitting a model to data. This makes sense, since the model is the dynamic object and the data is static (a.k.a. fixed and constant).

To put a point on it, I like Larry Wasserman's approach to this. In his telling, a statistical model is a collection of distributions. For example, the collection of all normal distributions:

$$ \{ \text{Normal}(\mu, \sigma) : \mu, \sigma \in R, \sigma > 0 \} $$

or the set of all Poisson distributions:

$$ \{ \text{Poisson}(\lambda) : \lambda \in R, \lambda > 0 \} $$

Fitting a distribution to data is any algorithm that combines a statistical model with a set of data (the data is fixed), and chooses exactly one of the distributions from the model as the one that "best" reflects the data.

The model is the thing that changes (sort of): we are collapsing it from an entire collection of possibilities into a single best choice. The data is just the data; nothing happens to it at all.

$\endgroup$
16
$\begingroup$

In the field of Rasch modelling it is common to fit the data to the model. The model is assumed to be correct and it is the analyst's job to find data which conform to it. The Wikipedia article on Rasch contains more details about the how and the why.

But I agree with others that in general in statistics we fit the model to the data because we can change the model but it is felt to be bad form to select or modify the data.

$\endgroup$
7
$\begingroup$

Typically, the observed data are fixed while the model is mutable (e.g. because parameters are estimated), so it is the model that is made to fit the data, not the other way around. (Usually people mean this case when they say either expression.)

When people say they fit data to a model I find myself trying to figure out what the heck did they do to the data?.

[Now if you're transforming data, that would arguably be 'fitting data to a model', but people almost never say that for this case.]

$\endgroup$
  • 5
    $\begingroup$ Removing outliers would also (arguably) be "fitting data to a model". $\endgroup$ – Federico Poloni Mar 24 at 8:34
  • 1
    $\begingroup$ The phrasing might make sense if they're thinking of it as "fitting (data to a model)". That is, you're doing a process of fitting, and that process of fitting starts from data and transforms it to a model. I agree that's a less common/accurate interpretation versus the "(fitting X) to Y" parse, but I put it out there as a rationale as to why someone might logically say it. $\endgroup$ – R.M. Mar 24 at 13:29
  • 1
    $\begingroup$ @FedericoPoloni Outliers are usually defined indepedently of the model that you later want to use. So even if we would want to call it fitting data, it would not be a model, but to something else. $\endgroup$ – BartoszKP Mar 24 at 20:16
  • 1
    $\begingroup$ +1. There is a reason it's called "data" - it is what is given, see the Latin origin of the word: latindictionary.wikidot.com/verb:dare $\endgroup$ – Christoph Hanck Mar 25 at 15:37
1
$\begingroup$

Usually, we assume our data corresponds to the "real world" and making any modifications means we are moving away from modelling the "real world". For example, one needs to take care removing outliers since even if it makes computation nicer, outliers were still part of our data.

When testing a model or estimating properties of an estimator using bootstrap or other resampling techniques, we may simulate new data using an estimated model and our original data. This makes the assumption that the model is correct, and we are not modifying our original data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.