I am following the course CS294-158  and got stuck with the first exercise that requests to implement the MADE paper (see here ). My implementation in TensorFlow  achieves results that are less performant than the solutions implemented in PyTorch from the course (see here ). I have been modifying hyperparameters there and around, trying to identify the main differences in the loss-functions, dimensionalities, etc. to no avail. I would kindly like to ask for help in understanding what I am doing wrong, or what I could make better to improve the performance. Maybe someone could explain to me what makes the PyTorch implementation better since I have just lost it. Thanks for your help!!
I have just found a difference in my implementation that makes the whole performance different: the random permutation of the attributes. According to the original paper, the permutation (order-agnostic training) should improve the performance in the density estimation, yet, I believe, if the training/testing compute an average over the permutations. I do not do any averaging. That downplayed the performance vs the PyTorch implementation...I believe this solves the question, but I leave it here for anyone who might want to throw a glance at my code. Thanks!