How many components to use in PCA in order to preserve a certain amount of variance? I want to reduce the dimensionality of my data with PCA, until it preserves $\alpha = 0.99$ of the variance. How do I decide how many eigenvectors I should use? 
So I'm looking for a function $f(data, \alpha) = k$ in order to use the first $k$ components of the projected data.
 A: Suppose $X$ is your data matrix. Using SVD you can find all its singular values $s_i$ and then calculate the cumulative sum of squared singular values. When the cumulated sum reaches $99\%$ of total sum, then you have the number, which means selecting the first $k$ PCs, you can preserve $99\%$ variance: $$\sum_{i=1}^{k} s_i^2 = 0.99 \sum s_i^2.$$
Matlab code:
X = bsxfun(@minus, X, mean(X));             %// center the data
[~,s,~] = svd(X);                           %// run SVD
normsqS = sum(s.^2);                        %// total variance
k = find(cumsum(s.^2)/normsqS >= 0.99, 1);  %// find k

A: You can reduce however many principal components you want...as long as you add a diagonal matrix to the k-reduced estimate you can match the variances exactly. For example, where $\mathbf{\widehat\Sigma}_{MOM}$ is the sample covariance matrix, we can write it as
$$
\mathbf{\widehat\Sigma}_{MOM} = \mathbf{V\Lambda V^T} = \sum_{i=1}^{p} \lambda_i\mathbf{v}_i\mathbf{v}_i^T
$$
Where $T$ is the transpose operator, $p$ the number of variables,$\mathbf{\Lambda}$ the diagonal matrix of eigen values,$\mathbf{V}$ a matrix of column eigen vectors and  $\mathbf{v}_i$ the $i^{th}$ eigen vector.  
Suppose I choose the first  $k<p$ principal components to be significant then I can form a new estimate
$$
\mathbf{\widehat\Sigma}_{NEW} = \sum_{i=1}^{k} \lambda_i\mathbf{v}_i \mathbf{v}_i^T+\mathbf{\Psi}
$$
where $\mathbf{\Psi}$ is a diagonal matrix constructed such that
$$
\mathrm{diag}(\mathbf{\widehat\Sigma}_{NEW}) = \mathrm{diag}(\mathbf{\widehat\Sigma}_{MOM}) 
$$
    This is a very common approach in many high dimensional problems and can be thought of as a "shrinkage" technique.  The problem with the answer given above is that it's a biased/inconsistent estimate of the variance (it will always underestimate the variance).  This shrinkage technique will not suffer from such bias and allows you to specify however many principal components you want to keep.  This comes in super handy when the rank of the covariance matrix is small.
There are a lot of different methods for choosing $k$, this Wikipedia page  highlights a few simpler ones.  Personally I use Monte Carlo simulations predicated on Parallel Analysis.  The method I use is robust to extreme rank deficiency, is reasonably quick, and predicts the correct number of significant principal components around 85% of the time at the 0.05 rejection level (and is usually pretty close when not exactly correct).  
If you want to be really quick you can use Random matrix Theory (RMT) techniques described 
 here  and
 here .  The last reference also includes interesting Bayesian techniques if your into that sort of thing.  RMT has a closed form solution, but it's based on asymptotics which may or may not hold up well with fewer observations or semi-definite covariance matrices.
