I'm still a bit confused about a few details of TPEs. So this should be a follow-up to this discussion: What is the "tree" structure in Tree Parzen Estimators?
See the original TPE paper here: https://proceedings.neurips.cc/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf

What is the connection between the tree-structured KDEs (in the inference graph) and the KDEs l(x) and g(x)?

My current understanding is as follows: TPE uses the tree-structured configuration space as a generative process. Each node is associated with a distribution from which a particular hyperparameter stems from. TPE replaces these distributions with individual KDEs which are then updated with observations to obtain the posteriors. My question is now: are l(x) and g(x) separate, "global" KDEs which are additionally defined to traverse the tree? Do individual KDEs represent l(x) and g(x), or am I missing something? I do not see the exact connection between l(x), g(x) and the KDEs in the tree-structure.
It may help to quantify how many KDEs there are. For the explanation above there are n + 2 KDEs. n is the number of nodes in the graph plus the KDEs l(x) and g(x).



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.