I'm trying to compare the two types of reduction by applying it to a list of ingredients for about 1000 sponge cakes.

The ingredient lists do however have miss certain ingredients out for some cakes, an example A small example is:

id   | flour | sugar | cherries | chocolate | ...
 1       300     20       50         100
 2       200     15        0         100

How is it best to proceed? Should I ignore columns which aren't fully populated? Should I perhaps classify these extras into a group where they're all populated? e.g if there was another column called apples, combine these with cherries?

I'm very new to this, any help would be appreciated greatly

  • $\begingroup$ I think that the correct approach would be to include all columns. For example if a cake doesn't have apples as its ingredient, add 0 to the apples for that row. What bothers me, however, is that you haven't shown to have a large number of ingredients (I don't know if you have more). I'm not sure if you'll be able to get good results for your dimensionality reduction techniques if you don't have many dimensions to begin with... $\endgroup$ – Djib2011 Apr 23 at 21:06
  • 1
    $\begingroup$ Thanks for the tip, I do have about 50 columns, I'll edit to make clearer $\endgroup$ – Dan Savage Apr 24 at 7:37
  • $\begingroup$ "Missing data" generally means that an entry in the data set isn't known. But, that doesn't seem to be the case here--If a recipe doesn't include an ingredient, it just means the value is zero. All values are known, so there's no need to do anything special. $\endgroup$ – user20160 Apr 24 at 12:25

You should certainly impute those values with zeros or data statistics if you are supposed to apply regular PCA. The same is valid for non-linear reduction tools, too.

However, if you are OK with applying some modifications of PCA which are robust to missing data, a brief literature tour gave me few articles about the subject. Among them this paper had some interesting notes. For instance, it illustrates a technique called GowPCoA which does not consider the missing values during computation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.