Is PCA selecting variables or constructing new variables? I know that PCA can be used for dimensionality reduction but there's something basic escaping me.
Let's say I have a feature set $F$ of $n$ features. After I run PCA, which of the following are true about the resulting feature set:


*

*It will be a subset of the original feature set $F$ with $k< n$ features where each of the remaining features was in the original set $F$.

*It will be a feature set with $k<n$ features where each feature is some new feature not in the original set $F$.


What I'm trying to get crystal clear is does PCA simply reduce the number of features in the original set to a smaller group that are just as predictive as the complete set or does it create new hybrid features from the original set?
 A: PCA creates new features that can be expressed as linear combinations of the old features.  Furthermore all the new features are orthogonal to each other, which prevents collinearity issues in regression.
You will always get a number of principal components less than or equal to the number of input features, which is where dimensionality reduction comes into play.  You can also discard the principal components that account for the least amount of variation in the original dataset.  A common threshold is to keep the PCs that account for 95% of the variation in the original dataset.
See also whuber's comment.  PCA can also destroy the predictive accuracy of your features.
A: It may be helpful to think of an intuitive example:
Say you're trying to model the term structure of interest rates, and you have a time series of interest rates for the front part of the treasury yield curve, years 2, 3, 5, and 10.  Without PCA, the number of dimensions you're looking at is 4.
Your data would look like:
[2y rate, 3y rate, 5y rate, 10y rate].
Two of the most common ways the yield curve can change are: 1. parallel shift and 2. change in slope.  Parallel shift is where everything goes up or down a similar amount, and change in slope is where the front increases and the back decreases, or vice versa.  These would be an intuitive example of what PCA literature would call "major axes of variation," and if you ran PCA on the yield curve data, you might see the first eigenvector being something like [0.5, 0.5, 0.5, 0.5] and the second being like [-0.7, 0, 0, 0.7].  
If you owned a portfolio of treasuries with weightings like [0.5, 0.5, 0.5, 0.5], the value of your portfolio would be sensitive to parallel shifts.  On the contrary, if you owned weightings like [-0.7, 0, 0, 0.7], your portfolio would be sensitive to changes in slope. This is the most important part of what I've said so far, so make sure it makes intuitive sense to you.  So now if you wanted to look at daily yield curve movement in terms of parallel shift and slope, you only have to consider 2 dimensions.
