Do we do data normalization just in case our initial data has gaussian distribution? We always do Data Normalization of our data when we have different ranges, and I found that normalization is just a translation followed by a multiplicative scaling, so it does not really change your distribution : a translated and scaled gaussian is still a gaussian ... But this is just in case our initial data is also gaussian, does it mean that if no gaussian data then we shouldn't do normalization? else we may lose our data variance because we change the whole distribution! Please if someone can clarify that foor me ...
 A: The z-score transformation is just a way to give your distribution a mean of zero and variance of one.
$$
z_i = \dfrac{x_i - \bar x}{s}
$$
That's the equation, no matter how the data are distributed.
You are correct to point out that distribution families are not always closed under the z-score transformation. For instance, if you subtract the mean of an exponential and then divide by the variance, you wind up with points less than zero, which are not allowed by any exponential distribution. However, the shape looks about the same. You can see this graphically.

library(moments)
library(ggplot2)
set.seed(2022)
N <- 1000
x <- rexp(N, 1)
z <- (x - mean(x))/sd(x)
d1 <- data.frame(x = x, Distribution = "Original")
d2 <- data.frame(x = z, Distribution = "Transformed")
d <- rbind(d1, d2)
ggplot(d, aes(x = x, fill = Distribution)) +
    geom_histogram() +
    facet_grid(~Distribution) +
    theme_bw()
mean(x) - mean(z)
sd(x) - sd(z)
moments::skewness(x) - moments::skewness(z)
moments::kurtosis(x) - moments::kurtosis(z)

Notably, metrics like skewness and kurtosis are not changed, except for some numerical technicalities in the $15$th and $16$th decimal places.
