You cited the Imbens & Rubin book in a comment...
From the introduction of chapter 7:
"The most important difference between the methods discussed in Chapters 5 and 6 and the ones discussed here is that they rely on different sampling perspectives. Both the Fisher approach discussed in Chapter 5 and the Neyman methods discussed in Chapter 6 view the potential outcomes as fixed and the treatment assignments as the sole source of randomness. In the regression analysis discussed in this chapter, the starting point is an infinite super-population of units. Properties of the estimators are assessed by resampling
from that population, sometimes conditional on the predictor variables including the treatment indicator. From that perspective, the potential outcomes in the sample are random..."
From the introduction of chapter 8:
"As discussed in Chapters 5 and 6, both Fisher’s and Neyman’s approaches for assessing treatment effects in completely randomized experiments viewed the potential outcomes as fixed quantities, some observed and some missing. The randomness in the observed outcomes was generated primarily through the assignment mechanism, and sometimes also through random sampling from a population. In this chapter, as in the preceding chapter on regression methods, we consider a different approach to inference, where the potential outcomes themselves are also viewed as random variables, even in the finite sample."
So it seems this is just a misunderstanding. Or where else did you read Rubin making this assumption of fixed potential outcomes (other than in explaining Fisher's and Neyman's work)?