Skip to main content
Commonmark migration
Source Link

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0). So, each datum has approximately 2~5 non-zero attribute values.

I want to visualize these data into 2-dimensional spaces. So my steps are like these.

  1. PCA process

    PCA process

    => let each datum get x, y coordinates.

  2. Clustering

    => DBSCAN, K-means, etc., something like those.

=> let each datum get x, y coordinates.

  1. Clustering

=> DBSCAN, K-means, etc., something like those.

I've heard that the proportion of variance is important, and I have the following proportions:

Importance of components: PC1    PC2    PC3    PC4     PC5     PC6    PC7     PC8     PC9     PC10 
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

(PC1's PV: 0.2009, PC2's PV: 0.1401)

So, when I convert data into 2-dimension space, as far as I've understood, I think I should project data into (PC1, PC2) coordinates, which only has 0.3410 (Cumulative Proportion)

Isn't 0.3410 (a slightly lower value than I'd expected) too unreliable for that data positioning? Also, is there other way to project that data into 2D space that has more cumulative proportion?

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0). So, each datum has approximately 2~5 non-zero attribute values.

I want to visualize these data into 2-dimensional spaces. So my steps are like these.

  1. PCA process

=> let each datum get x, y coordinates.

  1. Clustering

=> DBSCAN, K-means, etc., something like those.

I've heard that the proportion of variance is important, and I have the following proportions:

Importance of components: PC1    PC2    PC3    PC4     PC5     PC6    PC7     PC8     PC9     PC10 
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

(PC1's PV: 0.2009, PC2's PV: 0.1401)

So, when I convert data into 2-dimension space, as far as I've understood, I think I should project data into (PC1, PC2) coordinates, which only has 0.3410 (Cumulative Proportion)

Isn't 0.3410 (a slightly lower value than I'd expected) too unreliable for that data positioning? Also, is there other way to project that data into 2D space that has more cumulative proportion?

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0). So, each datum has approximately 2~5 non-zero attribute values.

I want to visualize these data into 2-dimensional spaces. So my steps are like these.

  1. PCA process

    => let each datum get x, y coordinates.

  2. Clustering

    => DBSCAN, K-means, etc., something like those.

I've heard that the proportion of variance is important, and I have the following proportions:

Importance of components: PC1    PC2    PC3    PC4     PC5     PC6    PC7     PC8     PC9     PC10 
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

(PC1's PV: 0.2009, PC2's PV: 0.1401)

So, when I convert data into 2-dimension space, as far as I've understood, I think I should project data into (PC1, PC2) coordinates, which only has 0.3410 (Cumulative Proportion)

Isn't 0.3410 (a slightly lower value than I'd expected) too unreliable for that data positioning? Also, is there other way to project that data into 2D space that has more cumulative proportion?

edited tags
Link
gung - Reinstate Monica
  • 147.5k
  • 89
  • 406
  • 717
edited title; added tag; formatted; edited for English; removed extra comments
Source Link
gung - Reinstate Monica
  • 147.5k
  • 89
  • 406
  • 717

Positioning Multivariate Data intomultivariate data in a 2-dimensional Spacespace (with PCA)

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0)

=>. So, each datum has approximately 2~5 non-zero attribute values...

I want to visualize these data into 2-dimensional spaces. So my steps are like these.

  1. PCA process

=> let each datum get x, y coordinates.

  1. Clustering

=> DBSCAN, K-means, ..etc. sth, something like those.

I'm newbie into PCA, but I've heard that the Proportionproportion of Variancevariance is important, butand I have belowthe following Proportions. (I got it with R programming)

Importance of componentsproportions: PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 Standard deviation 1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 Cumulative Proportion 0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

Importance of components: PC1    PC2    PC3    PC4     PC5     PC6    PC7     PC8     PC9     PC10 
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

(PC1's PV: 0.2009, PC2's PV: 0.1401)

So, when I convert data into 2-dimension space, as far as I've understood, I think I should project data into (PC1, PC2) coordinates, which only has 0.3410 (Cumulative Proportion)

Isn't 0.3410 (slightly lowa slightly lower value than I'd expected) too unreliable for that data positioning? Also, is there other way to project that data into 2D space that has more cumulative proportion?

Sorry for my bad background knowledge and English. I've struggled to find out by myself, but it is difficult.

Positioning Multivariate Data into 2-dimensional Space (with PCA)

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0)

=> So, each datum has approximately 2~5 non-zero attribute values...

I want to visualize these data into 2-dimensional spaces. So my steps are like these.

  1. PCA process

=> let each datum get x, y coordinates.

  1. Clustering

=> DBSCAN, K-means, ... sth like those.

I'm newbie into PCA, but I've heard that the Proportion of Variance is important, but I have below following Proportions. (I got it with R programming)

Importance of components: PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PC10 Standard deviation 1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 Cumulative Proportion 0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

(PC1's PV: 0.2009, PC2's PV: 0.1401)

So, when I convert data into 2-dimension space, as far as I've understood, I think I should project data into (PC1, PC2) coordinates, which only has 0.3410 (Cumulative Proportion)

Isn't 0.3410 (slightly low value than I'd expected) too unreliable for that data positioning? Also, is there other way to project that data into 2D space that has more cumulative proportion?

Sorry for my bad background knowledge and English. I've struggled to find out by myself, but it is difficult.

Positioning multivariate data in a 2-dimensional space (with PCA)

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0). So, each datum has approximately 2~5 non-zero attribute values.

I want to visualize these data into 2-dimensional spaces. So my steps are like these.

  1. PCA process

=> let each datum get x, y coordinates.

  1. Clustering

=> DBSCAN, K-means, etc., something like those.

I've heard that the proportion of variance is important, and I have the following proportions:

Importance of components: PC1    PC2    PC3    PC4     PC5     PC6    PC7     PC8     PC9     PC10 
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610 
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174 
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

(PC1's PV: 0.2009, PC2's PV: 0.1401)

So, when I convert data into 2-dimension space, as far as I've understood, I think I should project data into (PC1, PC2) coordinates, which only has 0.3410 (Cumulative Proportion)

Isn't 0.3410 (a slightly lower value than I'd expected) too unreliable for that data positioning? Also, is there other way to project that data into 2D space that has more cumulative proportion?

Source Link
Loading