I'm still a bit confused about a few details of TPEs. So this should be a follow-up to this discussion: What is the "tree" structure in Tree Parzen Estimators?
See the original TPE paper here: https://proceedings.neurips.cc/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf
What is the connection between the tree-structured KDEs (in the inference graph) and the KDEs l(x) and g(x)?
My current understanding is as follows: TPE uses the tree-structured configuration space as a generative process. Each node is associated with a distribution from which a particular hyperparameter stems from. TPE replaces these distributions with individual KDEs which are then updated with observations to obtain the posteriors. My question is now: are l(x) and g(x) separate, "global" KDEs which are additionally defined to traverse the tree? Do individual KDEs represent l(x) and g(x), or am I missing something? I do not see the exact connection between l(x), g(x) and the KDEs in the tree-structure.
It may help to quantify how many KDEs there are. For the explanation above there are n + 2 KDEs. n is the number of nodes in the graph plus the KDEs l(x) and g(x).
