Noise removal from a dataset with a know distribution If I have a dataset where it's points are drawn from a known distribution(For example a normal distribution) due to some noise the histogram doesn't reflect such a behavior (not necessarily skewed) Are there any algorithms that I can use to remove that noise so that the dataset retains it's distribution?
Note here that I'm not trying to normalize the dataset or produce a new derived feature with the normal distribution. Here I know the data is drawn from a normal distribution but due to some noise it's histogram doesn't reflect that fact. 
 A: I'm quite guessing here, but I will provide some answer.
Assuming you know you have normally distributed data with some covariance matrix $\Sigma$ and mean vector $\mu$, it is a good guess to assume you have additive Gaussian noise, with zero mean and independent from your data and between its samples (i.e. AWGN - additive white Gaussian noise). This is provided you see (according to your histograms) that the noise doesn't introduce skewness, shift in the mean, discrete peaks, etc.
So, you need to estimate the variance of the noise, $\sigma^2$. One way to do that is to use maximum-likelihood estimation:
$$ \hat{\sigma^2}=\arg\max_{\sigma^2} f(Y|\sigma^2)$$
Now, since $Y=X+N$ ($X$ for data, $N$ for noise), given $\sigma^2$, $Y\sim\mathcal{N}(\mu,\sigma^2I+\Sigma)$. The likelihood function will be a normal density:
$$f(Y|\sigma^2) = \frac{1}{\sqrt{(2\pi)^n\mbox{det}(\sigma^2I+\Sigma)}}\exp\left\{-\frac{1}{2}(Y-\mu)^T(\sigma^2I+\Sigma)^{-1}(Y-\mu)\right\}.$$
To obtain the estimator, you need to derive the above by $\sigma^2$ and find the maximum.
Once you have an estimation for the variance, you can do MAP or MMSE estimation. MAP will be given by:
$$\hat{X}=\arg\max_X P(X|Y)=\arg\max_X P(Y|X)P(X),$$
where $f(Y|X)=f(X+N|X)$ is just the noise density, shifted by the $X=x$ (given), and $f(X)$ is the Gaussian density for $X$. Similarly to the above, you derive, this time according to $X$, and obtain the estimation. Here, you use the estimation for $\sigma^2$ you found previously.
Another, maybe easier, option, is MMSE estimation, which due to the independent Gaussians turns out to be linear:
$$ E[X|Y] = E[X] + \Sigma_Y^{-1} \mbox{Cov}(X,Y) (Y-E[Y]). $$
