3
$\begingroup$

I read the following in the Google "rules for machine learning":

Rule #36: Avoid feedback loops with positional features.

The position of content dramatically affects how likely the user is to interact with it. If you put an app in the first position it will be clicked more often, and you will be convinced it is more likely to be clicked. One way to deal with this is to add positional features, i.e. features about the position of the content in the page. You train your model with positional features, and it learns to weight, for example, the feature "1st­position" heavily. Your model thus gives less weight to other factors for examples with "1st­position=true". Then at serving you don't give any instances the positional feature, or you give them all the same default feature, because you are scoring candidates before you have decided the order in which to display them.

I'm not I understand the advice. Why would anyone ever use positional features for ranking IF you don't have that information at serving time? If that's not feature that your system has at serving time, what value does it have to use it?

$\endgroup$
1
  • $\begingroup$ Any update on this one? I'd be also interested too! $\endgroup$
    – Weiye Deng
    Commented Oct 10, 2021 at 23:54

1 Answer 1

1
$\begingroup$

Imagine Search Engine Results Page (SERP). Obviously, position of a document affects its click probability a lot.

To do ranking, we want to predict document click probabiliy and sort values according to the scores. I.e. we need a model for P(Click | Query, Document).

On the other hand, the collected data is biased towards Position and looks like (Click, Query, Document, Position).

So:

  1. The collected data models P(Click | Query, Document, Position)
  2. Ranking model predicts P(Click | Query, Document)

Then the question is: how to train a position-free model on positional data? This problem is called debiasing.

One might say: let's just include Position as a feature during training and set Position=1 during serving. -- Rule #36 by Google calls it a bad idea.

Here is paper by Google about Position Bias Estimation: https://arxiv.org/pdf/2211.01494.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.