How to mathematically prove that PCA can reduce noise? We know $X^TX=U\Sigma U$ and reconstructed data $Y=X\tilde{U}$ for k eigenvector corresponding with k largest eigenvalue. Then how to mathematically show that PCA can reduce noise?
I am thinking about using covariance of Y to prove that. That is, the covariance of reconstructed data is less than the original data. Then, the reconstructed data contains less information so the noise is reduced. But I cannot relate less covariance in data and less noise in data using mathematical language. Can anyone help?
 A: First, as @Fiodor1234 mentioned, "less information" does not imply "less noise", and certainly not a lower ratio of noise to signal. Additionally, I don't think it is even clear what you mean by noise. Therefore, I intended to mention that there is no reason to believe PCA will reduce noise. However, I realized that that is not always true.
Consider a case where we draw observations with multiple features and

*

*The features have some degree of correlation

*The features are recorded with random measurement errors that are IID.

In this case, because PCA exploits correlation structure in the data, reconstructing the dataset using some subset of PCs will indeed yield a dataset that is less influenced by measurement error than the original. Consider this simulated example where data is drawn from a multivariate gaussian distribution and recorded with random normal errors. The measurements are reproduced by the PCs much more faithfully (higher $R^2$) than the measurement errors.

import numpy as np
from numpy.random import multivariate_normal
from sklearn.datasets import make_spd_matrix
from sklearn.decomposition import PCA
from sklearn.metrics import r2_score
import seaborn as sns
from matplotlib import pyplot as plt
import pandas as pd

np.random.seed(777)

results ={
    'nComp': [],
    'Measure': [],
    'R2': []
}
N_obs = 1000
N_features = 20
for i in range(15):
    cov = make_spd_matrix(N_features)
    mean = np.array([0]*N_features)
    X = multivariate_normal(mean, cov, [N_obs])
    noise = np.random.normal(0,0.6, [N_obs, N_features])
    for nComp in range(1, N_features+1):
        pca = PCA()
        pca.fit(X+noise)
        X_hat = np.dot(pca.transform(X)[:,:nComp], pca.components_[:nComp,:])
        noise_hat = np.dot(pca.transform(noise)[:,:nComp], pca.components_[:nComp,:])
        X_noise_hat = np.dot(pca.transform(X+noise)[:,:nComp], pca.components_[:nComp,:])
        r2_X = r2_score(X,X_hat)
        r2_noise = r2_score(noise,noise_hat)
        r2_X_noise = r2_score(X+noise, X_noise_hat)
        results['nComp'] += [nComp]*3
        results['R2'] += [r2_X, r2_noise, r2_X_noise]
        results['Measure'] += ['True X', 'Measurement Error', 'Observed X']
        
results_df = pd.DataFrame(results)
plt.figure(figsize=(9,6))
sns.pointplot(data=results_df,x='nComp',y='R2', hue='Measure')
plt.ylim(0)
plt.ylabel("$R^2$ Score - Reconstruction vs Original")
plt.xlabel("Number of PCs")
plt.title("Proportion of Variance Explained \nvs Number of Principal Components")
plt.show()

