# distribution of image data?

(I don't much about deep learning, but have been playing with a few things and have some questions.)

I took the (pretrained on imagenet) resnet18 model from pytorch, removed the last fully-connected layer and ran the STL-10 images (96 pixels square) through it. The result is a 512-dimensional data set. It seems that this data is spherical in that the variance in norms is $$\approx0.01$$ and all of the data is in a spherical shell of radius between 20 and 22. It also seems that none of the features are redundant as seen on this PCA explained variance curve:

I did something similar, running the CIFAR-10 images (32 pixels square) through the pretrained densenet121 model from pytorch (with the last linear layer removed), getting 1024-dimensional data. This data doesn't seem to be spherical at all, and there is a lot of redundancy among the features:

I assume the observations above come from the architecture of the models, which I know nothing about really. My questions are:

1. Are there any rules of thumb about the distribution of image data run through some of the popular networks?
2. Is there a reason why the resnet produces spherically distributed data?
3. Is there a reason why the densenet produces a lot of dependent data?

(Note: To use pytorch models, images were resized to 256 pixels square and center-cropped at 224 pixels square.)

• You should maybe compare the two curves on the same scale? – Jon Nordby Feb 10 at 7:54
• In (3) you say that you resized the images. Are you making small images larger? What size are the images before and after resizing? What method do you use for resizing? – Sycorax Feb 10 at 16:44
• If you want to compare the two models, I would recommend using the same data. – Jon Nordby Feb 10 at 18:41