Potential for bias to be introduced when limiting analysis to participants who seek treatment? I am running an online AB test where users are assigned to condition (test or control) at login, but only 1/3 of assignees reach the feature being tested.
Analyzing all assignees will give me an accurate estimate of the effect of releasing the test version of the feature in the full user population, but this intent-to-treat analysis will also water down the estimate of the new feature's effectiveness among actual users, a separate but important question.
So if I analyze only the 1/3 of users who actually use of the feature (a per protocol analysis) is it possible I am introducing bias into my analysis, bias that would not be protected against by random assignment to condition? Is there any reason to think that my sample of feature users is different than if I had randomized at the moment they reached the feature rather than at login?
Assume there is no way that users would know they were in a given condition, there is no way for users to access the alternative condition, and there are equal rates of the feature's use in both conditions. Is there any reason not to assume sequential ignorability: That use of the tested feature was still as random as assignment to condition? Have I possibly introduced unmeasured confounding when I perform the per protocol analysis?
 A: Estimating the per-protocol effect does not introduce confounding bias because there are no confounders. I think a reasonable DAG based on your description might be:
$$A \rightarrow F \rightarrow Y \leftarrow U \rightarrow \boxed{R}$$
where $A$ is assignment to treatment, $F$ is which version of the feature is actually seen), $Y$ is your outcome, $R$ is whether someone reached the feature, and $U$ is the unmeasured causes of the outcome and the failure to reach the feature (e.g., interest in the product). Conditioning on $R$ by only looking at those who reached the feature does not introduce any noncausal associations between $F$ and $Y$; no backdoor paths are opened and no frontdoor paths are blocked.
The main issue is that your estimand is different when you condition on reaching the feature; it is now a conditional, rather than marginal, estimand. It generalizes only to those who would be inclined to reach the feature, and not to those who would not be inclined to reach the feature. If the effect of the feature differed, for example, between interested users and uninterested users, then the estimated effect among those who reached the feature would not represent the effect of the feature in the total population of users. In your case, this is actually desirable; unless you force all users to reach the feature, you really are only interested in the effect of the feature on those who reach the feature.
If the DAG is incorrect, then it is possible for biases to be introduced, even in estimating the conditional effect. If users were clued into to which feature they might receive ($A$) and that influenced whether they reached it or not (i.e., adding the path $A \rightarrow U$ or $A \rightarrow R$), conditioning on reaching the feature ($R$) would induce collider bias. As you have described it, this is not an issue. The only issue is the population to which the effect generalizes.
