Sampe Standard deviation, what distribution does it follow? Suppose we have $k$ groups of $n$ i.i.d observations which follow the $N(0,1)$ parent distribution. The sample standard deviation $S$ is a random variable (with $k$ available observations), do we know what distribution does it follow? I also have the same question when the parent distribution is $N^2(0,1)$, which basically means that the parent distribution is a $\chi^2_1$.
 A: It is easier to discuss the distribution of the sample variance
$$S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2,$$ where
$X_i$ are a random sample from $\mathsf{Norm}(\mu, \sigma)$ and
$\bar X$ is the sample mean.
In that case,
$$\frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(\nu = n-1).$$ 
Thus, the distribution of $S^2$ is a multiple of $\mathsf{Chisq}(\nu = n-1).$
The second displayed relationship can be proved using a multivariate transformaion
or by using moment generating functions. [After an appropriate linear transformation of an $n$-dimensional multivariate normal distribution, $\bar X$ has a one dimensional marginal distribution and, independently, $S^2$ is a function of $n-1$ dimensions.]
You don't say why you seek the distribution of $S.$ The relationship above, using $\mathsf{Chisq}(\nu = n-1),$ can be used to find a 95% confidence interval for $\sigma$
as follows. 
First, a 95% CI for $\sigma^2$ is of the form 
$\left(\frac{(n-1)S^2}{U},\frac{(n-1)S^2}{L}\right),$ where $L$ and $U$ cut probabiltiy
0.025 from the left and right tails, respectively, of $\mathsf{Chisq}(\nu = n-1).$
Then, to find a 95% CI for $\sigma,$ take square roots of the endpoints of the above
CI for $\sigma^2.$
Of course, the distribution of $S$ can be found by taking the square root of the
relevant chi-squared distribution. But historically, printed tables of percentage points of chi-squared distributions have been available, so the distribution of $S^2$ is more
commonly used. 
Notes: (1) For normal data $E(S^2_n) = \sigma^2,$ but because of the linear nature of
expectation, this equality does not survive taking square roots: $E(S_n) = \sigma\sqrt{\frac{2}{n-1}}\Gamma(\frac{n}{2})/\Gamma(\frac{n-1}{2}) < \sigma,$ where $S_n$ is the standard deviation of a sample of size $n.$ The bias of $S_n$ as an estimator of $\sigma$ is small; except for very small samples. For samples of moderate or large size, the bias is often ignored in practice. For example: $E(S_5) = 0.9400\sigma$ and $E(S_{50}) = 0.9949\sigma.$
n = c(5,25,50);  round(sqrt((2/(n-1)))*gamma(n/2)/gamma((n-1)/2),4)
[1] 0.9400 0.9896 0.9949

(2) For a random sample from the standard normal distribution, the distribution of $S_{25}$
is simulated below:
set.seed(2020)
s = replicate( 10^6, sd(rnorm(25)) )
summary(s)
  Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
0.4191  0.8905  0.9862  0.9897  1.0850  1.7311 
hist(s, prob=T, br=30, col="skyblue2", main="Dist'n of Sample SD")


A: The relevant distribution here is called the chi distribution:
$$S \sim \frac{\sigma}{\sqrt{n-1}} \cdot \text{Chi}(\text{df} = n-1).$$
Using the rules for transformations of random variables, the density function for the standard deviation is:
$$\begin{aligned}
f_S(s) 
&= \text{Chi} \Bigg( \frac{\sqrt{n-1} \cdot s}{\sigma} \Bigg| \text{df} =  n-1 \Bigg) \cdot  \frac{\sqrt{n-1}}{\sigma} \\[6pt]
&= \frac{(n-1)^{n/2}}{2^{(n-3)/2} \cdot \sigma \cdot \Gamma(\tfrac{n-1}{2})} \cdot \Big( \frac{s}{\sigma} \Big)^{n-2} \cdot \exp \Big( - \frac{n-1}{2} \cdot \frac{s^2}{\sigma^2} \Big). \\[6pt]
\end{aligned}$$
The resulting mean and variance are:
$$\begin{aligned}
\mathbb{E}(S) 
&= \sigma \cdot \sqrt{\frac{2}{n-1}} \cdot \frac{\Gamma(\tfrac{n}{2})}{\Gamma(\tfrac{n-1}{2})}, \\[6pt]
\mathbb{V}(S) 
&= \sigma^2 \Bigg[ 1 - \frac{2}{n-1} \cdot \frac{\Gamma(\tfrac{n}{2})^2}{\Gamma(\tfrac{n-1}{2})^2} \Bigg]. \\[6pt]
\end{aligned}$$
