2
$\begingroup$

I have created a dataset of pictures taken at the museum of different paintings. The dataset is divided into 113 different categories (paintings) and contains around 4.8k images.

Just to be clear: these pictures were taken for our research by our team, these images are not taken from users who tested the app (which is still in development). The purpose of this dataset is to test our approach in a realistic way, following a realistic distribution, using realistic devices and taking the images like different users would do. Changing the dataset in such a context is legit and I don't think that there is something wrong with it.

I want that this dataset fits a power-law. In order to do so, I'll check if the log-log plot, where on the x axis there bill be the paintings in order of popularity (so x = 1 is the most popular painting in the dataset) and f(x) will be the fraction of images in the dataset relative to that painting.

Probably, the resulting log-log plot will not be a straight line, so the dataset will probably not fit a power-law distribution.

I have two questions:

  1. Given the x-th most popular painting, how many pictures I have to add/remove so my dataset follows a power-law? The only solution that came to my mind is to draw a trending line represented by g(x) and query it for each x so I know how many pictures to add/remove for each painting.
  2. As alternative, let's suppose that I want that my dataset have 113 categories (so the x axis has 113 values) and I want that my dataset has 5k images, how many pictures do I have to take for each category so the resulting dataset fits a power-law?

Why I need this:

We are developing a cache for virtual reality applications. It has been proven how the subjects of multimedia applications usually follows a power-law distribution. In our case we are developing two applications: a cinema movie poster recognition and a painting recognition. Caches strategies are effective only if the subjects follow a power-law distribution (and so a cache hit is probable). Up to now it doesn't exist such a datasets to simulate our caching strategy (so pictures of paintings/posters taken at the museum/cinema with smartphones following a power-law) so we are creating it (and make it public, it's part of our contribution). I hope this make more clear why I need this.

$\endgroup$

closed as unclear what you're asking by Kodiologist, Michael Chernick, mdewey, Nick Cox, Scortchi Apr 9 '17 at 18:51

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 4
    $\begingroup$ Why would you want to modify your dataset so that it fits a particular model? Is this inverse statistics? $\endgroup$ – Olivier Apr 7 '17 at 1:36
  • 2
    $\begingroup$ user6321, you're going to be stuck in an XY problem until you can bring yourself to explain what you're trying to accomplish. $\endgroup$ – Kodiologist Apr 7 '17 at 2:44
  • 1
    $\begingroup$ Regardless of what happens with similar variables in other data sets, I see no reason to adjust your data. $\endgroup$ – Peter Flom Apr 8 '17 at 11:26
  • 2
    $\begingroup$ Everything is wrong with it. You are lying about your data. $\endgroup$ – Peter Flom Apr 8 '17 at 11:33
  • 2
    $\begingroup$ @PeterFlom I appreciate that you're contributing, but you're not giving reasons about because this is wrong. It's not very constructive, I'm sorry. $\endgroup$ – user6321 Apr 8 '17 at 15:45