Sample Size vs Iterations (CLT) I am slightly confused by the term 'sampling size'. Let's say we have N-dimensional distribution and we take one sample (i.e., one vector of N-elements) and measure it's mean. Now let's do 100 iterations and now we have 100 vectors (each of N-elements). My question here is: does the sample size refer to N or 100? By the way, I am asking this question in the context of CLT. 
 A: From what I understand so far, you have the following setup: for $i = 1, \dots, 100$, you have
$$X_i = (X_{i1}, X_{i2}, \dots, X_{iN}) \overset{iid}{\sim} F $$
where $F$ is a $N$-dimensional distribution. I am further making the assumption that each of the $X_{ij}$ are iid. That is the pdf of $F$ is $\prod_{j=1}^{N} f_j$ where $f_j$ are all the same. Let the mean of each $X_{ij}$ be $\mu$ and the variance be $\sigma^2$.
Then the mean of each $N$-dimensional vector is $$\bar{X}_i = \dfrac{1}{N} \sum_{j=1}^{N} X_{ij} ,$$ and by the CLT
$$\sqrt{N}(\bar{X}_i - \mu) \overset{d}{\to} N(0, \sigma^2). $$
This CLT equation means that we would expect the sample means of the $N$ dimensional vectors to follow a pattern according to that equation. If $N$ is large enough, then this equation will hold approximately, and thus
$$\bar{X}_i \approx N\left(\mu, \dfrac{\sigma^2}{N}\right). $$
Thus, the sample size would refer to $N$ here, and not 100. The number 100 refers to the size of repeated samples.
This in turn means that each of the $\bar{X}_1, \bar{X}_2, \dots, \bar{X}_{100}$ is an approximate sample from a $N(\mu, \sigma^2/N)$ distribution. This is termed as the sampling distribution of the sample mean. The exactness of the result depends on how large $N$ is. If originally $X_{ij}$ was a normal distribution, the result holds exactly, and not approximately. 
In the below example, I set $N$ = 1000 and let $X_{ij} \sim N(0,1)$. Below on the left is the density plots, with black being the density from 100 such samples and red being the density of the $N(0, 1/1000)$ distribution. The density plot looks better if instead of 100 draws, I do 1000, as is seen in the plot on the right.

I do this again with $N$, but $X_{ij} \sim \text{Gamma}(2,1)$. Then the sampling distribution for the sample means is $N(2, 2/N)$ and thats what I have plotted. Note how the approximation is not as good in this case.

In practice, we rarely have the liberty of having obtained 100 replicated of a sample, and thus generally we obtain a sample of size $N$, find the sample mean, and see where it lies on the sampling distribution. Thus, the term "sampling distribution" refers to the theoretical distribution of the sample mean. Repeated sampling from would lead to multiple sample means, which tend to line up according to the sampling distribution (just like in the plots above).
