23
$\begingroup$

I'm writing an example and have made up some data. I want it to be clear to the reader this is not real data, but I also don't want to give the impression of malice, since it just serves as an example.

There is no (pseudo)random component to this particular data, so it seems to me that 'simulated' is not appropriate. If I call it fictitious or fabricated, does that give the impression of fraudulent data? Is 'made-up' a word that would fit in a scientific context?

What is the terminology in statistical literature for non-simulated made-up data?

$\endgroup$
  • 9
    $\begingroup$ Just to add a comment which spreads across several answer: "synthetic" is a good word for made up data which tries to look as realistic as possible, while "mock up" suggests data which has been crafted to demonstrate something particular. For example, "mock up" data might contain absurd outliers, just to demonstrate how important it is to deal with outliers properly. $\endgroup$ – Cort Ammon Aug 4 at 20:02
  • $\begingroup$ I personally prefer the term "simulated" and have encountered it the most in statistical literature (i.e., "we conducted simulations to compare our model vs. X,Y,Z...." $\endgroup$ – Samir Rachid Zaim Aug 26 at 13:23

10 Answers 10

45
$\begingroup$

I would probably call this "synthetic" or "artificial" data, though I might also call it "simulated" (the simulation is just very simple).

$\endgroup$
  • 30
    $\begingroup$ One hears "toy data," "toy example," and "dummy data." Also I agree that "simulated" might well fit even in the absence of random numbers. $\endgroup$ – rolando2 Aug 4 at 9:33
  • 7
    $\begingroup$ "Illustrative data" or "example data" might also work $\endgroup$ – Henry Aug 4 at 9:44
  • 8
    $\begingroup$ +1 'synthetic data' and 'toy example' are both terms I might use, if the occasion arose, as is 'constructed example'. Sometimes I say "illustrative example" or something similar, particularly when the example was explicitly constructed to have particular features (e.g. when designed as a counterexample to some mistaken notion). $\endgroup$ – Glen_b Aug 4 at 10:10
  • 1
    $\begingroup$ I tend to use toy data (without artificial or simulated) for real (measured) data sets that I "abuse" to demonstrate something. $\endgroup$ – cbeleites supports Monica Aug 5 at 17:27
  • 1
    $\begingroup$ It depends a bit on your application what will work best. For example, I am also doing a project with "fake" data, but another part of the project involves using a computer model simulation. So it might confuse the reader for me to refer to the fake data as "simulated", falsely implying the data come from the simulation. So I've been relying on "artificial", and at times I describe the data as "manufactured". I personally would avoid "synthetic" as to me this term would imply that the data is some sort of combination of other data sources (a "synthesis" of e.g. data A and data B). $\endgroup$ – Ceph Aug 6 at 20:26
12
$\begingroup$

If you want to refer to your data as fictitious you'd be in good company, as that's the term Francis Anscombe used to describe his now famous quartet.

From Anscombe, F. J. (1973). "Graphs in Statistical Analysis", Am. Stat. 27 (1):

Some of these points are illustrated by four fictitious data sets, each consisting of eleven (x, y) pairs, shown in the table.

But I think your caution is well placed, as my OED (v4) seems to indicates that this use of fictitious is obsolete

fictitious, a.

(fɪkˈtɪʃəs)

[f. L. fictīci-us (f. fingĕre to fashion, feign) + -ous: see -itious.]

1.1 †a.1.a Artificial as opposed to natural (obs.). b.1.b Counterfeit, ‘imitation’, sham; not genuine.

$\endgroup$
  • $\begingroup$ In terms of readability the first suggestion & the comments are much better alternative. No need to use uncommon, complicated words. $\endgroup$ – Tim Aug 4 at 14:38
  • 1
    $\begingroup$ @Tim: I want to agree, but I'm not entirely sure what I'd be agreeing with. Are you saying that fictitious would be a bad choice, despite having been used in a similar context before? Because that's what I'm saying. $\endgroup$ – AkselA Aug 4 at 14:46
7
$\begingroup$

In IT we often call it mockup data, which can presented through a mockup (application).

The mockup data can also be presented through a fully functional application, for instance to test the functionality of the application in a controlled manner.

$\endgroup$
  • 5
    $\begingroup$ Good point, but I believe that mockup data and simulated data are not exactly the same. When creating mockup data for unit tests, you need it only to preserve some very basic properties of the real data, while when using simulated data for statistical analysis, you usually use more sophisticated data examples. $\endgroup$ – Tim Aug 4 at 20:02
  • 2
    $\begingroup$ I still believe ErikE is correct though, when you write analytical code you either need the real thing or mock data. Mock data can be as big as you want it to be imo. $\endgroup$ – Mathijs Segers Aug 6 at 5:52
  • 1
    $\begingroup$ Practices probably vary as does use of terminology, I guess. For many of our tests and analyses we use live data which has been "defused" for reasons of security and anonymity. For others we create bare bones data just as Tim describes. I have no strong opinion but we do use the term mockup quite loosely. $\endgroup$ – ErikE Aug 6 at 10:22
3
$\begingroup$

I've seen repeated suggestions for the term "synthetic data". That term however has a broadly used, and very different meaning from what you want to express: https://en.wikipedia.org/wiki/Synthetic_data

I am not sure there is a generally accepted scientific term, but the term "example data" seems hard to misunderstand?

$\endgroup$
  • 1
    $\begingroup$ That article seems a little confused--the relationship to anonymization is pretty tenuous. $\endgroup$ – Matt Krause Aug 6 at 16:29
  • $\begingroup$ +1 but I agree with previous comment: apart from the second paragraphs (saying that synthesized data is a type of anonymized data), the rest of that Wikipedia article does seem to be describing what the questioner wants. I.e. realistic-looking made-up data. $\endgroup$ – Darren Cook Aug 9 at 11:09
2
$\begingroup$

I've encountered the term 'fake data' a fair amount. I guess it could have some negative connotations but I've heard it often enough that it doesn't register negatively at all for me.

FWIW, Andrew Gelman uses it too:

https://statmodeling.stat.columbia.edu/2009/09/04/fake-data_simul/

https://statmodeling.stat.columbia.edu/2019/03/23/yes-i-really-really-really-like-fake-data-simulation-and-i-cant-stop-talking-about-it/

https://books.google.dk/books?id=lV3DIdV0F9AC&pg=PA155&lpg=PA155&dq=fake+data+simulation&source=bl&ots=6ljKB6StQ4&sig=ACfU3U17GLP_84q_HfIQB4u5O6wV0yA2Aw&hl=en&sa=X&ved=2ahUKEwiF2_eB0uvjAhWswcQBHSn5Cn04ChDoATAAegQICRAB#v=onepage&q=fake%20data%20simulation&f=false

A quick google search for 'fake data' turns up a lot of results that seem to be using the term similarly:

https://scientistseessquirrel.wordpress.com/2016/03/10/good-uses-for-fake-data-part-1/

http://modernstatisticalworkflow.blogspot.com/2017/04/an-easy-way-to-simulate-fake-data-from.html

https://clayford.github.io/dwir/dwr_12_generating_data.html

And there's even a fakeR package, which suggests that this is relatively common: https://cran.r-project.org/web/packages/fakeR/fakeR.pdf

$\endgroup$
2
$\begingroup$

I use a different word depending on the manner in which I use the data. If I have found the made-up dataset lying around and have pointed my algorithm at it in a confirmatory manner, then the word "synthetic" is fine.

However, oftentimes whenever I use this type of data, I have invented the data with the specific intent of showing off the capabilities of my algorithm. In other words, I invented data for the specific purpose of getting "good results". In such circumstances, I am fond of the term "contrived" along with an explanation of my expectations for the data. This is because I don't want anyone to make the mistake of thinking that I pointed my algorithm at some arbitrary synthetic dataset I found lying around and it really worked out well. If I have cherry-picked data (to the point of actually making it up) specifically to make my algorithm work out well, I say so. This is because such results provide evidence that my algorithm can work out well, but provide only very weak evidence that one might expect the algorithm to work out well in general. The word "contrived" really sums up nicely the fact that I have chosen the data with "good results" in mind, a priori.

"does that give the impression of fraudulent data?"

No, but, it is important to be clear about the source of any dataset and your a priori expectations as the experimenter when reporting your results on any dataset. The term "fraud" explicitly includes an aspect of having covered something up or having outright lied. The #1 way to avoid commission of fraud in science is to simply be honest and forthright about the nature of your data and your expectations. In other words, if your data are fabricated and you fail to say as much in any way, and there is some kind of expectation that the data are not fabricated or, worse, you claim that the data are gathered in some non-fabricated sort of way, then that is "fraud". Don't do that thing. If you want to use some synonym for the term "fabricated" that "sounds better", such as "synthetic", nobody will fault you, but at the same time I don't think that anyone will notice the difference except for you.

A side note:

Less obvious are circumstances where one claims to have had a priori expectations that are actually post hoc explanations. This is also fraudulent analysis of data.

There is a danger of this when one chooses data specifically with the intent of "showing off" the capabilities of an algorithm, which is frequently the case with synthetic data.

To be clear about why this is the case, consider that the "normal" scientific method works something like so: 1) A population $D$ is chosen 2) A hypothesis $H$ is concieved 3) $H$ is tested against $D$ (or some sample chosen from $D$). Science doesn't have to work within this narrow definition, but this is what is called "confirmatory" analysis, and is generally considered the strongest form of evidence one can provide. Since the order of events correlates with the strength of evidence, it is important to specifically document them.

Notably, in the case of "contrived" data, the process often works more like so: 1) A hypothesis $H$ is conceived, 2) A population $D$ is chosen, 3) $H$ is tested against $D$. If you are testing an algorithm, for example, then the hypothesis that your fancy new algorithm "does a good job" might occur prior to the invention of the synthetic dataset. If this is the case, you should mention it. At the very least you should not purport that events transpired in a "confirmatory" manner, because that would lead readers to conclude that your evidence is stronger than it actually is.

There is no problem with doing this, so long as you are honest and forthright about what you have done. If you have gone through pains to create a dataset that gives "good results", do say so. As long as you let the reader know the steps that you have taken in your data analysis, they have the information necessary to effectively weigh the evidence for or against your hypotheses. When you are not honest or are not forthright, then this may give the impression that your evidence is stronger than it really is. When you are KNOWINGLY less than honest and forthright for the sake of making your evidence seem stronger than it really is, then that is, indeed, fraudulent.

In any case, this is why I prefer the term "contrived" for such datasets, along with a short explanation that they are, indeed, chosen with a hypothesis in mind. "Contrived" conveys the sense that not only did I create a synthetic dataset, but I did so with particular intentions that reflect the fact that my hypothesis was already in place before the creation of my dataset.

To illustrate by an example: You create an algorithm for analysis of arbitrary time-series. You hypothesize that this algorithm will give "good results" when pointed at time-series. Consider, now, the following two possibilities: 1) You create some synthetic data that looks like the sort of thing that you expect your algorithm to perform well on. You analyze this data and the algorithm performs well. 2) You grab some synthetic datasets because they are available why not. You analyze this data and the algorithm performs well. Which of these two circumstances provides the better evidence that your algorithm performs well on arbitrary time-series? Clearly, it is option 2. However, it might be easy to report in either option 1 or option 2 that "we applied algorithm $A$ to synthetic dataset $D$. Results are shown in Figure $x.y$." In the absence of any context, a reader might reasonably assume that these results are confirmatory (option 2), when, in the case of option 1, they are not. The reader has therefore, in option 1, been given the impression that the evidence is stronger than it really is.

tl;dr

Use whatever term you like, "synthetic", "contrived", "fabricated", "fictitious". However, the term that you use is insufficient to ensure that your results are not misleading. Ensure that you are clear in your report about how the data came about, including your expectations for the data and the reasons why you chose the data that you chose.

$\endgroup$
  • $\begingroup$ Although the answers here overlap and almost all make good points this one I think best conveys the key point that no single term will convey to all readers the intent behind making up data. The reaons can range from not just appropriate but essential for the purpose through laziness (poor introductory texts) to cheating and fraud. Explaining why you are doing it at some length may be a good idea. $\endgroup$ – Nick Cox Aug 26 at 9:54
  • $\begingroup$ ... reasons ... $\endgroup$ – Nick Cox Aug 26 at 11:05
1
$\begingroup$

First, there's no reason to not call it a "dataset". There is no universally agreed upon term(s) for "fake" vs "simulated" vs ... data. If the goal is to be completely clear, it's best to actually devote a sentence, rather than a word, to qualify what this dataset is. After that, you can relax the designation and just refer to your data as data.

"Synthetic", "artificial" does not distinguish from other MCMC sampled "simulated" datasets in my mind. Using a quasirandom number generator with a fixed seed (as proper training would dictate) also creates a synthetic or artificial dataset.

If the point of curating a dataset for a specific illustration, rather than generating an instance or realization from a probability model, I think it's better to call such a dataset an "example dataset". Data like these are akin to Anscombe's quartet: totally abstract and not-plausible, but meant to illustrate a point.

$\endgroup$
1
$\begingroup$

In biology, analyses are sometimes demonstrated using a dataset of mythical animals. Whether or not to explicitly state that the data are simulated is up to the author/reviewer.

An ecologist’s guide to the animal model, 2009

These tutorials describe a series of quantitative genetic analyses on a population of gryphons (reflecting a compromise between the avian and mammalian biases of the authors). As the gryphon is a mythical beast the data provided were necessarily simulated.

Fixed effect variance and the estimation of repeatabilities and heritabilities:Issues and solutions, 2017

To illustrate this, let us go back to Wilson (2008)’s unicorn dataset. It is a known fact that in unicorns, horn length varies according to the individual body mass (slope: β = 0.403 for a full model including age, sex and their interaction).

$\endgroup$
  • 1
    $\begingroup$ Interesting approach! I think this could be great for teaching biology students statistics. When presenting to the public though, I'm not sure whether this would give off the right impression $\endgroup$ – Frans Rodenburg Aug 28 at 23:46
0
$\begingroup$

Intuitively I would go to the term 'Dummy data', in the same sense that "Lorem ipsum..." is called 'Dummy text'. The word 'Dummy' is quite general and easy to understand for people from various backgrounds and is therfore less likely to be misinterpreted by readers of a less statistical background.

$\endgroup$
  • 2
    $\begingroup$ If it's in a regression context, I would avoid overloading "dummy", lest you have dummy variables encoding dummy data. $\endgroup$ – Matt Krause Aug 5 at 21:24
  • $\begingroup$ I agree, I would personally avoid it since "Dummy" already has a set connotation in regression. Given that there are an abundance of terms available, it's probably best to avoid those terms that can mean different things for different people. $\endgroup$ – Samir Rachid Zaim Aug 26 at 16:35
0
$\begingroup$

Data is Latin for given, that is used in modern times as a shorthand for given set of recorded facts. So in a way referring to fabricated recordings as some sort of given facts would be an open contradiction.

However, due to the increasing use of data to refer simply to recordings - regardless of the original presumption of records being of facts - we happily understand each other when talking about recordings that may or may not be truthful - hence real/fake data.

I will summarise my experience of ways to address fabricated recordings below. The label used depends whether one is assuming that we are talking of data as fabricated recordings that are intended to look reasonably realistic to enable further analysis, or data as a computational load.

  • In analytics/data science/strategic consultancies circles, people address most frequently a fabricated set of recordings generated under realistic assumptions as synthetic data - and occasionally simulated data. Fabricated recordings created using crude assumptions are referred to as toy dataset.
  • Among software engineers, fake data, dummy data, made-up data and mock-up data are frequent labels that mainly hint to recordings not necessarily meant to have realistic properties, but only share basic properties with the original data (age data is always numerical, email addresses always strings that contain “@“).
  • Academic researchers would refer to a realistic set of fabricated recordings as pseudo-data, or simulated data. In some circles, if the fabricated set of observations is the result of a Monte Carlo simulation, it may be referred to colloquially as Monte Carlo. Semi-realistic recordings are commonly used for illustrative purpose or testing alternate hypotheses, and referred to as toy dataset
$\endgroup$
  • 2
    $\begingroup$ "Monte Carlo" is the name of the method, so the "colloquial" name would be very misleading. $\endgroup$ – Tim Aug 5 at 21:30
  • $\begingroup$ @Tim indeed, it may be seen as misleading. However, language is just a tool grounded on consensus in a community as a way to refer to something. So much so that we are referring on this site to recordings and measurements as given (English for Latin data). If I were to adopt your viewpoint, I would find addressing simulated measurements as fake given highly questionable. $\endgroup$ – famargar Aug 26 at 9:09
  • $\begingroup$ I hope you will see now that referring to a “Monte Carlo simulation” as simply “Monte Carlo” is a modern version of referring to “given observations” as “given”. I edited my answer to incorporate this and more considerations of the meaning vs actual usage of the word “data”. $\endgroup$ – famargar Aug 26 at 10:17
  • 1
    $\begingroup$ "Academic researchers would refer to a realistic set of fabricated recordings most frequently as pseudo-data": I don't recall ever seeing this term in 40+ years of academic research. "Academics typically have no use for unrealistic recordings": sorry, but that strikes as quite wrong. Academics in many, many fields use simulations of several different kinds. Even unrealistic simulations can be useful, e.g. the variability of normal samples is important context for assessing non-normality. $\endgroup$ – Nick Cox Aug 26 at 11:08
  • $\begingroup$ @NickCox Pseudodata is frequently used in physics, and I have seen it in biology and statistics. Would be curious to know what’s your field and how your field refers to simulations. As for unrealistic data, I made a distinction between unrealistic and semi-realistic. Did I miss your use case? $\endgroup$ – famargar Aug 26 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.