2
$\begingroup$

I am reading the Yoshua Bengio et al, Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation. The objective function, Equation (11) and (12) are set for a given trajectory $\tau$. I have the following questions.

  1. I suppose a trajectory is a sequence $(s_1,a_1,s_1,a_2,\cdots,s_h)$. Is that correct?

  2. Is $\tau$ sampled randomly?

  3. At each state node $s'$ on the trajectory $\tau$ there are state nodes $s$ connected to (either incident onto or emanated from) $s'$ but does not belong to the trajectory such that $F(s,a)=s'$ or $F(s',a)=s$. But neither $F$ is constrained by the equation (11) or (12). In other words, there are "free dangling" nodes along the trajectory not constrained by Equation (11) or (12), which implies Equation (11) and (12) do not constrain the flow. Now, if we further impose a constraint such as all flows incident onto or emanating from a node are all equal, or that the dangling flows are all zero, then the constraint is effective. Or more simply and directly, we could just limit the set of state action pairs $(s,a)$ to those that are pre-set to be explored instead of those are possible. The paper seems to be ambiguous on this point. What am I missing?

$\endgroup$

1 Answer 1

0
$\begingroup$

first author here.

  1. Yes

  2. $\tau$ can be sampled arbitrarily as long as we have adequate coverage. A sensible choice is to follow a policy $\pi$, with a higher temperature to encourage exploration, or, as in the paper, take a random action with probability $\ll 1$.
    In the paper the policy is constructed from the log-flows as softmax(logits=log_F) (which achieves $p(x)\propto R(x)$). In the new Foundations paper, we propose some ways to explicitly parameterize the policy.

  3. Perhaps I am misunderstanding the question, but in the flow matching objective of (11) & (12) we do sum over all the parents and all the children of each state in the trajectory (all the states $s$ such that $T(s,a)=s'$ and $T(s',a)=s$, which I assume you mean here instead of $F$--we used $F$ for flows and $T$ for the transition).
    We also expect this to work without having to do a learning step on all states. The idea is that we hope to learn an assignment of the flow that generalizes well, so we can not only do well on the training set, but also an unseen test set.
    That being said, the point that there are unlimited solutions here holds, but it's possible to constrain things such that there is a unique solution. In the Foundations paper, section 5.2 explains that a "backwards policy" $P_B$ can be chosen freely and that $P_B$ and $R$ together completely and uniquely specify the flow $F$.

$\endgroup$
1
  • 1
    $\begingroup$ I am fortunate to get the information directly from the horse's mouth! Thank you very much, Manux, for all the detailed valuable information! Glancing over the Foundations paper, it appears to be much more rigorous than the conference paper I cited. While I currently think you may have somewhat misunderstood my Question 3, please allow me to read the Foundations paper first and come back to clarify my question. $\endgroup$
    – Hans
    Dec 6, 2021 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.