This topic is actually rather hard to google for as 'tree' has been overloaded in this domain to refer to decision trees.
I'd be interested in having a learning algorithm produce code, such as used in Microsoft Power BI's feature to query databases using natural language ('Q&A', formerly 'InfoNav').
Evidently, natural language and database queries can both be expressed as text, so we have a good answer: recurrent neural networks.
However, if we take at SQL queries for one, then intuitively the actual prediction job can be simplified if we'd look at their AST representation rather than their textual one.
If we were to predict an AST representation, language keywords would suddenly no longer clog up the output vector to be predicted, as we could use the BNF describing the language and have the decisions at each node be represented as an integer or the like, along with other info (e.g. the content of number/string literals).
Is there a standard way to approach this? What are best practices here?
Wikipedia has an article on structured prediction, but obviously it's based on theory and academics, and even articles featured are fairly old (for ML), so I'm inclined to think it may not necessarily tell me much about current best practice, if any...
A recursive neural network (RNN) is a kind of deep neural network created by applying the same set of weights recursively over a structure, to produce a structured prediction over variable-size input structures, or a scalar prediction on it, by traversing a given structure in topological order. RNNs have been successful for instance in learning sequence and tree structures in natural language processing, mainly phrase and sentence continuous representations based on word embedding.
$\endgroup$