Can Reinforcement Learning be used for generative 2D mechanism design? As my msc. thesis I am working on a generative design algorithm for the design of mechanisms (in the mechanical engineering sense). I'm specifically interested in the design of compliant mechanisms.
In current solutions only topology optimizations (TO) are used, or evolutionary algorithms (EA). TO is very designer-dependent (which I'd rather not see: I want the tool to be an enhancement to human creativity). EA works, but it seems non-ideal, since its basically stochastic.
So I was thinking: can I use machine learning to develop some sort of intuition during the generative design process. Supervised and unsupervised learning seem incompatible, since no data exists. Besides, learning from human-designed mechanisms kind of defeats the purpose.
What we do have, however, are methods to evaluate mechanism designs. So it is possible to assess a designs performance. This is what brought me to reinforcement learning (RL): we may be able to create a reward signal for mechanisms.
RL however seems mostly concerned with sequential decision making, which is atypical for mechanism design. In literature, reference was found to a method describing mechanisms by a series of operators that 'build' a mechanism. Such a description might be seen as a sequential process.
My question: could RL be used to perform generative mechanism design? If not, other alternatives from machine learning would be appreciated!
 A: In principle, as long as you have a reward signal, yes. You could consider mechanical mechanism design as the case of making sequential decisions about the parameters of each component. However, you don't observe rewards (the return) until the end of an "episode" (the sequence of decisions that fully specifies your mechanism). This is fine, it just makes the "credit assignment" problem harder (which actions led to which rewards).
If the action space is large (many parameters required to fully specify a design), you could use policy-gradient reinforcement learning to approach this problem. Policy-gradient reinforcement learning uses the observed returns and "pushes" the policy to make the actions that yielded higher returns more likely. If the evaluation of each mechanism is efficient, then the method should work well - it can quickly obtain experience. You could try using a Recurrent Neural Network-based policy gradient method, and will have to design 1) the action representation 2) the observations the policy sees at each decision step 3) the reward signal
A: Turns out it is possible. Please read about my results in this paper.
