Objective functions of the flow network based generative model by Yoshua Bengio?

Question

I am reading the Yoshua Bengio et al, Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation. The objective function, Equation (11) and (12) are set for a given trajectory $\tau$. I have the following questions.

1. I suppose a trajectory is a sequence $(s_1,a_1,s_1,a_2,\cdots,s_h)$. Is that correct?

2. Is $\tau$ sampled randomly?

3. At each state node $s'$ on the trajectory $\tau$ there are state nodes $s$ connected to (either incident onto or emanated from) $s'$ but does not belong to the trajectory such that $F(s,a)=s'$ or $F(s',a)=s$. But neither $F$ is constrained by the equation (11) or (12). In other words, there are "free dangling" nodes along the trajectory not constrained by Equation (11) or (12), which implies Equation (11) and (12) do not constrain the flow. Now, if we further impose a constraint such as all flows incident onto or emanating from a node are all equal, or that the dangling flows are all zero, then the constraint is effective. Or more simply and directly, we could just limit the set of state action pairs $(s,a)$ to those that are pre-set to be explored instead of those are possible. The paper seems to be ambiguous on this point. What am I missing?

Answer 1

first author here.

1. Yes

2. $\tau$ can be sampled arbitrarily as long as we have adequate coverage. A sensible choice is to follow a policy $\pi$, with a higher temperature to encourage exploration, or, as in the paper, take a random action with probability $\ll 1$.
   In the paper the policy is constructed from the log-flows as softmax(logits=log_F) (which achieves $p(x)\propto R(x)$). In the new Foundations paper, we propose some ways to explicitly parameterize the policy.

3. Perhaps I am misunderstanding the question, but in the flow matching objective of (11) & (12) we do sum over all the parents and all the children of each state in the trajectory (all the states $s$ such that $T(s,a)=s'$ and $T(s',a)=s$, which I assume you mean here instead of $F$--we used $F$ for flows and $T$ for the transition).
   We also expect this to work without having to do a learning step on all states. The idea is that we hope to learn an assignment of the flow that generalizes well, so we can not only do well on the training set, but also an unseen test set.
   That being said, the point that there are unlimited solutions here holds, but it's possible to constrain things such that there is a unique solution. In the Foundations paper, section 5.2 explains that a "backwards policy" $P_B$ can be chosen freely and that $P_B$ and $R$ together completely and uniquely specify the flow $F$.