I am following the course CS294-158 [1] and got stuck with the first exercise that requests to implement the MADE paper (see here [2]). My implementation in TensorFlow [3] achieves results that are less performant than the solutions implemented in PyTorch from the course (see here [4]). I have been modifying hyperparameters there and around, trying to identify the main differences in the loss-functions, dimensionalities, etc. to no avail. I would kindly like to ask for help in understanding what I am doing wrong, or what I could make better to improve the performance. Maybe someone could explain to me what makes the PyTorch implementation better since I have just lost it. Thanks for your help!!
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I have just found a difference in my implementation that makes the whole performance different: the random permutation of the attributes. According to the original paper, the permutation (order-agnostic training) should improve the performance in the density estimation, yet, I believe, if the training/testing compute an average over the permutations. I do not do any averaging. That downplayed the performance vs the PyTorch implementation...I believe this solves the question, but I leave it here for anyone who might want to throw a glance at my code. Thanks!