How does the direction of maximum variance change with a linear transformation? If I apply a transformation matrix $R$ to a distribution of sample points $D$, does the direction of maximum variance $k$ change in a definable way? By definable I mean can an equation be written in terms of $k$ and $R$ and possibly other things that can predict the new direction of maximum variance?
My original instinct was to think that $k_\mathrm{new} = Rk$ or something, but after thinking about it, this would not be the case, because the transformation could change the variance in different parts of the distribution.
I'm familiar with Principal Component Analysis, but would love an explanation about what is typically expected when transformations are applied.
 A: The mapping $f(R): k \mapsto k_\mathrm{new}$ is nonlinear and discontinuous. There probably isn't any analytical expression for it.
As an example, consider a two-dimensional dataset $D$ with a covariance matrix $$C=\left(\begin{array}{cc}4&0\\0&1\end{array}\right),$$ e.g. data coming from a 2D Gaussian with the variance in $x$ equal to $4$, variance in $y$ equal to $1$, and zero correlation between $x$ and $y$. The first PC $k$ (I am using your notation here) points in $x$ direction: $k=(1,0)$.
Now let's apply transformation $$R=\left(\begin{array}{cc}a&0\\0&1\end{array}\right)$$ that is "compressing" or "stretching" the data ellipse horizontally. Set $a=1$ and start gradually decreasing it. The ellipse will compress, but as long as $a>1/2$ it will remain horizontally elongated,  hence $k$ will not change: $k_\mathrm{new}=k$. At $a=1/2$ the covariance becomes spherical, i.e. $k_\mathrm{new}$ is undefined. As soon as $a<1/2$, the ellipse becomes vertically stretched, hence $k_\mathrm{new} = (0,1)$.
We see that $k_\mathrm{new}$, as a function of $R$, does not change continuously: $k_\mathrm{new}$ can suddenly jump elsewhere with a small change in $R$. Hence $k \mapsto k_\mathrm{new}$ definitely cannot be a linear function (proving your original intuition wrong), and most likely there is no analytical expression for it whatsoever.
