What do eigenvectors of a data matrix consisting of house features/prices tell us? [duplicate]

Question

I know this is one of the most repetitive question but bear with me please. I am trying to gain an intuitive understanding of eigenvectors. I had this example in my mind where there is a matrix A, the columns correspond to features of house such as floors, rooms etc and rows correspond to houses. Now if I compute the eigenvectors of this matrix what information will those eigenvectors give me ?

Answer 1

What you're really asking is, given a data matrix $X$ containing different features, i.e. some columns/features are categorical, some are real valued, some are price, some might be dummy variables, what would the eigenvalues of the data matrix's covariance matrix $\Sigma$ tell us about the data contained in $X$.

Since principal component analysis (PCA) is the most common learning algorithm that uses eigenvalues, PCA would give an approximation to the "intrinsic" dimensionality of $X$ based on its $\Sigma$ and nothing more since PCA is a dimensionality reduction tool. PCA will not directly tell you which features are the most interesting ones, it will only give a number of principal components and their "strengths" (eigenvalues of the covariance matrix of $X$).

If the task is feature selection on the housing data, there are more considerations, prior to eigenvalue analysis, that have to be taken into account. With the columns/features being very distinct from one another statistically (categorical/numerical, etc) some thought has to be put into which of the features you really care about that at the same time are 'readable' by eigenvector analysis. If $X$'s features had all been numerical and real-valued, you could go directly to looking at the eigenvalues, maybe by only having to standardize the features first, but with the case of housing datasets there will likely be alot of pre-cleaning involved, even omission of several of the columns in $X$ in order for the eigenvectors to deal with the irregular information structure.