1
$\begingroup$

I have data about ~18.000 files, mostly name, and size. I would like to plot it with Mathplotlib. Not sure what would be the best option.

df_final.head(2)
                       file_sizes_clean                    number_of_files
folder1/               [36829.0, 38711.0, 0, 349159, ...]   9
folder2/               [71318.0, 46944.0,... ]              6

Collecting all the file sizes into a single np.array:

from scipy import stats
x = np.array(sizes_flat)
stats.describe(x)

DescribeResult(nobs=18664, minmax=(3336.0, 8876007.0), 
mean=280316.77716459497, variance=445060834502.10297, 
skewness=4.744572944478072, kurtosis=29.465744857449934)

Trying to plot it:

f, axes = plt.subplots(1, 1, sharey=True, figsize=(20, 8))
axes.set_yscale('log')
sns.distplot(x, hist=True, kde=False, bins=100, color = 'blue', hist_kws={'edgecolor':'black'})

enter image description here

Different plot:

f, axes = plt.subplots(1, 1, sharey=True, figsize=(20, 8))
axes.set_yscale('log')
plt.plot(x,x)

enter image description here

I was wondering if there is a better way of plotting these or it would make sense to try to take a different approach, maybe go by subfolders. I would like to display the extreme skewness in the data that we have many very small files and some medium-sized. Ideally, we would have roughly equal-sized files.

$\endgroup$
  • 1
    $\begingroup$ A treemap also seems to be an interesting option. The filesize determines the surface area. Color could e.g. depend on subfolder (or filetype). $\endgroup$ – JohanC Feb 6 at 23:30
  • $\begingroup$ This is an excellent recommendation. Thanks! $\endgroup$ – Istvan Feb 7 at 18:05
2
$\begingroup$

Choice of plots depends on what you want to communicate to the reader, so it is always a contextual issue. Assuming that your goal is simply to give the reader a sense of the distribution of the file sizes, a histogram of the data is perfectly reasonable. Once you add clear axis titles and a plot title, the histogram you have constructed ought to be sufficient to give the reader a clear idea of the distribution of the data.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is what I thought. I might just split the data into smaller chunks and display histograms for those chunks as well. Thank you! $\endgroup$ – Istvan Feb 4 at 8:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.