I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but unfortunately I have to apply it to a system whose covariance matrix has multiple collinearities.
I've found a way to bypass this issue by using dimensionality reduction. Again, if the covariance matrix is nonsingular, the basis vectors for the transformed space come from the eigendecomposition of the covariance matrix. If the covariance matrix has multiple collinearities, you can perform a singular value decomposition on it instead. This is where I get stuck.
My question has multiple parts:
How does dimensionality reduction transform the training problem of comparing log-probability ratios to simply comparing Euclidean distances from class centroids?
How is using SVD in the transformation process equivalent to the eigendecomposition method?
How can I calculate conditional probabilities after I've transferred over to using the SVD method?
Question 3 is the most important for me at the moment because I am trying to implement this in C++. Question 1 & 2 are to better my own understanding of the problem.