I want to know whether Agglomerative Hierarchical clustering draws non-convex cluster boundaries. From sklearn's comparing diff clustering algorithm experiment it seems like Agglomerative clustering can form non-convex clustering even though it is a distance based method. I don't understand how that's possible.
Yes, it produces non-convex clusters.
You have an example in your screenshot.
But also no: it does not "draw cluster boundaries". There is no concept of a cluster boundary in hierarchical clustering. Clusters are just sets of points.
In particular single-linkage tends to form "chains" that can be non-convex. Since it uses nearest neighbors only, it ignores the overall shape of the cluster.