How to conduct am A/B test for a feature which cannot be accessed by every visitor?

Question

I am prepping for an interview and here is one of the questions I have come across:

How would you conduct an A/B test on an opt-in feature?

I think you cannot perform a "fair" test with an opt-in option. People viewing the A version of the site will come from the whole distribution of users whereas people viewing B will only come from the set who have opted in. This seems to violate the conditions for a controlled experiment.

What is your take on this?

Answer 1

Regarding your statement

You are correct: If the test shall compare an opt-in version of a site to a non-opt-in-version, then one cannot perform a fair test. Since visitors how have opted-in are probably more engaged than regular users, I'd expect the group of opt-in visitors to outperform the other group in engagement related metrics as long as meaningful versions are compared (although such a test may uncover that the opt-in version is not good enough). It might even be that the metrics for different versions are not comparable at all (because of different goals).

Another interpretation of the interview question

_{Since you have used the self-study-tag, I'd provide some starting points for further thinking instead of a complete answer (I can answer them myself if requested though, otherwise I suggest that you provide an answer to your own question).}

Here are two more interpretations of the interview question

1. How to conduct an A/B-Test for a feature that shall lead to an opt-in (i.e. the visitors have not opted in yet but the feature shall make them do so) ? This would be a more standard A/B-Test setup so I guess it is unlikely the question was meant this way.
2. How to conduct an A/B-Test restricted to test different versions of an opt-in feature. This would be my first guess how the question is meant (more complicated). So let's focus on this one.

So how to conduct an A/B-Test for the second case ? Consider that the site with the opt-in feature can be in an restricted area (i.e. you have to e.g. log in to get there) or can be just a different version of a site every visitor can access (e.g. the homepage).