2
$\begingroup$

I'm playing with a "minor" variation on an otherwise typical low rank matrix factorization collaborative filtering algorithm. I'm mostly following Andrew Ng's description in Coursera's online ML course - with this "minor" variation.

I'm given:

  • A reasonable amount of user preference data
  • A sparse feature data set

I want to compute the User parameter vectors (Theta), using the provided feature data, and use them to predict the full set of user preference values.

Example:

        UserA  UserB  UserC ... UserN | Feature1 F2 F3 F4 F5 F6 ... FN
item_1    0      5      ?         ?   | 1        0  NA NA NA NA     NA
item_2    1      ?      ?         5   | NA       NA 1  1  NA NA     NA
item_3    5      0      2         0   | NA       NA NA NA 0  1      0
...                                   |
itemN     0      0      5         0   | 0        1  NA 1  NA NA     NA

Not all features apply to all items. Each item has 10-100 features worth of applicable data, out of a total of say 1000 features.

My question is how I should process the NA (Not applicable) features. Most of the feature data is boolean (and where it isn't, presume it's normalized).

  • Should I treat the NA's as 0's (false)?
  • Should I give NA's their own, otherwise unused (-1) value?
  • Or am I wading into dangerous territory with this modification to the typical format of the algorithm.
  • Perhaps an approach I'm not thinking of?

Data/problem example:

  • Item #1 is a shape. A green square to be specific.
  • Item #2 is a feeling. A blue sadness (we're associating colors with feelings these days)

Our feature set then is:

         x0   x1         x2         x3          x4      x5    x6    x7       x8
            isSquare  isCircle  isTriangle  isHappy  isSad  isRed isGreen isBlue
item#1    1    1          0          0          NA      NA     0     1        0
item#2    1   NA         NA         NA           0       1     0     0        1

For most items we've got good quality feature data like this, sure, we'd like to make use of user preference data to help us improve our feature data, but for the most part we want to derive user preferences from existing feature data.

Hence I'm trying to implement the process of calculating preference data from features separately from calculating features from preferences, in hopes of controlling that process where I know the feature data is highly accurate (but still being able to learn feature data that might be inaccurate or to identify questionable assumptions in the feature data).

I'd like to play with the process some to see how a few assumptions I have play out. But I'm perplexed by the NA's in my feature set.

My best thought so far is that they're just 0's, as a feeling "is NOT a circle", which is essentially a true statement, as irrelevant as the comparison might be.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ One way is to treat them as missing values and just directly look for a low-rank approximation w/o imputing some preset value: stats.stackexchange.com/a/35460/13669 stats.stackexchange.com/a/35476/13669 $\endgroup$
    – Stumpy Joe Pete
    Commented Dec 17, 2014 at 23:57
  • $\begingroup$ If I understand you correctly, that would amount to basing the recommendation entirely on the user preference data right? The "typical" algorithm learns the features entirely on its own. However the feature data in this case has been very meticulously crafted by subject matter experts, we want the recommendation results to derive significantly from the feature data (later I want to play with some back and forth between calculating features from preferences and vice versa where our feature data might be incomplete, or contain questionable assumptions). $\endgroup$
    – David Parks
    Commented Dec 18, 2014 at 0:04

1 Answer 1

Reset to default
1
$\begingroup$

NA is not 0. In the nomenclature from the class, your $R$ matrix would reflect whether it is NA or not. For your $Y$, it doesn't matter what you put there because after multiplying with $R$, it won't matter anyway.

Improve this answer
$\endgroup$
3
  • $\begingroup$ But my $R$ matrix indicates what ratings the Users have provided, right? That part of it's well documented. Should I apply a second $R$ matrix for the features that aren't applicable? That's the part that is throwing me off now. The material I'm basing this off of assumes that feature data is complete for all items,features. $\endgroup$
    – David Parks
    Commented Dec 17, 2014 at 22:28
  • $\begingroup$ To clarify nomenclature, for all readers, the $R$ matrix referenced contains 1|0's indicating which items/users R(i,u) have a rating (1) and which don't (0). $\endgroup$
    – David Parks
    Commented Dec 17, 2014 at 23:48
  • $\begingroup$ I am not sure I understand. You only need one $R$ matrix. All you are doing is using it to compute the error (and thus gradient) for values that the user has rated. If they are not, your error matrix will not end up with those anyway. This is user - item level, not feature level. $\endgroup$
    – ignorant
    Commented Dec 29, 2014 at 19:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.