Return to Answer

Please study both my links in the comments. The 1st gives the LDA extraction algorithm formulas, and the 2nd (in my answer there) contains the demonstration of LDA on iris dataset according to such computations. I am not using R's term "scaling matrix", finding it a bit vague.

Per the algorithm expressed in my 1st link, LDA produces eigenvalues (which we recalculate into canonical correlations) and eigenvectors. Eigenvectors - we can multiply them by the constant $\sqrt{N-k}$ to obtain the (raw, unstandardized) discriminant coefficients $\bf C$ (called by you "unnormalized scaling matrix") which are the coefficients to compute discriminant scores with. Alternatively, we could normalize the eigenvectors to column SS=1, to obtain the matrix $\bf C_n$ (you call it "normalized scaling matrix") usable, for visual purposes, as the (oblique) rotation matrix of the variables into the discriminants.

So, you may use either way to get the values (scores) of the discriminants ($\bf X$ being the centered analyzed variables): one via discriminat coefficients: $\bf XC$, another via normalized eigenvectors: $\bf XC_n$. The first set of scores have the property that their pooled within-class covariance matrix is the identity matrix. The second set of scores are the perpendicular projections of the data points onto the discriminants as axes drawn in the space of the variables. By both sets of scores, the discriminants are uncorrelated variates. But, as drawn in the space of the original variables, they are not orthogonal axes.

By tradition, the first way to compute discr. scores is used most often by packages.

[As far as I'm aware, R and Python do not follow the fast algorithm I described under my link. They probably use equivalent, slower yet computationally more stable, variant utilizing SVD instead of eigendecomposition. Again, I'm mentioning that issue in my linked answer.]

Please study both my links in the comments. The 1st gives the LDA extraction algorithm formulas, and the 2nd (in my answer there) contains the demonstration of LDA on iris dataset according to such computations. I am not using R's term "scaling matrix", finding it a bit vague.

Per the algorithm expressed in my 1st link, LDA produces eigenvalues (which we recalculate into canonical correlations) and eigenvectors$^1$. Eigenvectors - we can multiply them by the constant $\sqrt{N-k}$ to obtain the (raw, unstandardized) discriminant coefficients $\bf C$ (called by you "unnormalized scaling matrix") which are the coefficients to compute discriminant scores with. Alternatively, we could normalize the eigenvectors to column SS=1, to obtain the matrix $\bf C_n$ (you call it "normalized scaling matrix") usable, for visual purposes, as the (oblique) rotation matrix of the variables into the discriminants.

So, you may use either way to get the values (scores) of the discriminants ($\bf X$ being the centered analyzed variables): one via discriminat coefficients: $\bf XC$, another via normalized eigenvectors: $\bf XC_n$. The first set of scores have the property that their pooled within-class covariance matrix is the identity matrix. The second set of scores are the perpendicular projections of the data points onto the discriminants as axes drawn in the space of the variables. By both sets of scores, the discriminants are uncorrelated variates. But, as drawn in the space of the original variables, they are not orthogonal axes.

By tradition, the first way to compute discr. scores is used most often by packages.

[As far as I'm aware, R and Python do not follow the fast algorithm I described under my link. They probably use equivalent, slower yet computationally more stable, variant utilizing SVD instead of eigendecomposition. Again, I'm mentioning that issue in my linked answer.]

$^1$ Because $\bf{S_w^{-1} S_b}$ of LDA isn't symmetric, its eigenvectors does not have to be an orthonormal matrix, unlike eigenvectors of PCA.

Please study both my links in the comments. The 1st gives the LDA extraction algorithm formulas, and the 2nd (in my answer there) contains the demonstration of LDA on iris dataset according to such computations. I am not using R's term "scaling matrix", finding it a bit vague.

Per the algorithm expressed in my 1st link, LDA produces eigenvalues (which we recalculate into canonical correlations) and eigenvectors. Eigenvectors - we can multiply them by the constant $\sqrt{N-k}$ to obtain the (raw, unstandardized) discriminat coefficients $\bf C$ (called by you "unnormalized scaling matrix") which are the coefficients to compute discriminant scores with. Alternatively, we could normalize the eigenvectors to column SS=1, to obtain the matrix $\bf C_n$ (you call it "normalized scaling matrix") usable, for visual purposes, as the (oblique) rotation matrix of the variables into the discriminants.

So, you may use either way to get the values (scores) of the discriminants ($\bf X$ being the centered analyzed variables): one via discriminat coefficients: $\bf XC$, another via normalized eigenvectors: $\bf XC_n$. The first set of scores have the property that their pooled within-class covariance matrix is the identity matrix. The second set of scores are the perpendicular projections of the data points onto the discriminants as axes drawn in the space of the variables. By both sets of scores, the discriminants are uncorrelated variates. But, as drawn in the space of the original variables, they are not orthogonal axes.

By tradition, the first way to compute discr. scores is used most often by packages.

[As far as I'm aware, R and Python do not follow the fast algorithm I described under my link. They probably use equivalent, slower yet computationally more stable, variant utilizing SVD instead of eigendecomposition. Again, I'm mentioning that issue in my linked answer.]

Please study both my links in the comments. The 1st gives the LDA extraction algorithm formulas, and the 2nd (in my answer there) contains the demonstration of LDA on iris dataset according to such computations. I am not using R's term "scaling matrix", finding it a bit vague.

Per the algorithm expressed in my 1st link, LDA produces eigenvalues (which we recalculate into canonical correlations) and eigenvectors. Eigenvectors - we can multiply them by the constant $\sqrt{N-k}$ to obtain the (raw, unstandardized) discriminant coefficients $\bf C$ (called by you "unnormalized scaling matrix") which are the coefficients to compute discriminant scores with. Alternatively, we could normalize the eigenvectors to column SS=1, to obtain the matrix $\bf C_n$ (you call it "normalized scaling matrix") usable, for visual purposes, as the (oblique) rotation matrix of the variables into the discriminants.

So, you may use either way to get the values (scores) of the discriminants ($\bf X$ being the centered analyzed variables): one via discriminat coefficients: $\bf XC$, another via normalized eigenvectors: $\bf XC_n$. The first set of scores have the property that their pooled within-class covariance matrix is the identity matrix. The second set of scores are the perpendicular projections of the data points onto the discriminants as axes drawn in the space of the variables. By both sets of scores, the discriminants are uncorrelated variates. But, as drawn in the space of the original variables, they are not orthogonal axes.

By tradition, the first way to compute discr. scores is used most often by packages.

[As far as I'm aware, R and Python do not follow the fast algorithm I described under my link. They probably use equivalent, slower yet computationally more stable, variant utilizing SVD instead of eigendecomposition. Again, I'm mentioning that issue in my linked answer.]

Please study both my links in the comments. The 1st gives the LDA extraction algorithm formulas, and the 2nd (in my answer there) contains the demonstration of LDA on iris dataset according to such computations. I am not using R's term "scaling matrix", finding it a bit vague.

Per the algorithm expressed in my 1st link, LDA produces eigenvalues (which we recalculate into canonical correlations) and eigenvectors. Eigenvectors - we can multiply them by the constant $\sqrt{N-k}$ to obtain the (raw, unstandardized) discriminat coefficients $\bf C$ (called by you "unnormalized scaling matrix") which are the coefficients to compute discriminant scores with. Alternatively, we could normalize the eigenvectors to column SS=1, to obtain the matrix $\bf C_n$ (you call it "normalized scaling matrix") usable, for visual purposes, as the (oblique) rotation matrix of the variables into the discriminants.

So, you may use either way to get the values (scores) of the discriminants ($\bf X$ being the centered analyzed variables): one via discriminat coefficients: $\bf XC$, another via normalized eigenvectors: $\bf XC_n$. The first set of scores have the property that their pooled within-class covariance matrix is the identity matrix. The second set of scores are the perpendicular projections of the data points onto the discriminants as axes drawn in the space of the variables. By both sets of scores, the discriminants are uncorrelated variates. But, as drawn in the space of the original variables, they are not orthogonal axes.

By tradition, the first way to compute discr. scores is used most often by packages.

[As far as I'm aware, R and Python do not follow the fast algorithm I described under my link. They use equivalent, slower yet computationally more stable, variant utilizing SVD instead of eigendecomposition. Again, I'm mentioning that issue in my linked answer.]

Please study both my links in the comments. The 1st gives the LDA extraction algorithm formulas, and the 2nd (in my answer there) contains the demonstration of LDA on iris dataset according to such computations. I am not using R's term "scaling matrix", finding it a bit vague.

Per the algorithm expressed in my 1st link, LDA produces eigenvalues (which we recalculate into canonical correlations) and eigenvectors. Eigenvectors - we can multiply them by the constant $\sqrt{N-k}$ to obtain the (raw, unstandardized) discriminat coefficients $\bf C$ (called by you "unnormalized scaling matrix") which are the coefficients to compute discriminant scores with. Alternatively, we could normalize the eigenvectors to column SS=1, to obtain the matrix $\bf C_n$ (you call it "normalized scaling matrix") usable, for visual purposes, as the (oblique) rotation matrix of the variables into the discriminants.

So, you may use either way to get the values (scores) of the discriminants ($\bf X$ being the centered analyzed variables): one via discriminat coefficients: $\bf XC$, another via normalized eigenvectors: $\bf XC_n$. The first set of scores have the property that their pooled within-class covariance matrix is the identity matrix. The second set of scores are the perpendicular projections of the data points onto the discriminants as axes drawn in the space of the variables. By both sets of scores, the discriminants are uncorrelated variates. But, as drawn in the space of the original variables, they are not orthogonal axes.

By tradition, the first way to compute discr. scores is used most often by packages.

[As far as I'm aware, R and Python do not follow the fast algorithm I described under my link. They probably use equivalent, slower yet computationally more stable, variant utilizing SVD instead of eigendecomposition. Again, I'm mentioning that issue in my linked answer.]