0
$\begingroup$

I am reading the PyTorch documentation on using LSTM to classify names with a character-level RNN and generating names with a character-level RNN.

In the examples they show, the network's hidden_size is set to 128. Why is that? How do they know that 128 makes the learning work? In general, is there a guiding principle (besides trial and error) that we may use to set this parameter's value? Will 128 work in general and transfer to other problem domains?

Also, I find that it is not clear why they define the classification and generation networks architectures as so. If we can transform our problem to the ones like they've demonstrated, will the network architectures be reasonable and/or applicable to use?

I realized that nearly identical questions (here and here) have been asked, but those are in general, and here, I am referring to concrete examples.

$\endgroup$

1 Answer 1

1
$\begingroup$

How do they know that 128 makes the learning work?

They probably tried a number of other configurations (say, 32, 64, 256, 512) units and found that 128 worked well. (It's possible that using more units works better, but they went with 128 for some other reason, such as speed of training.)

In general, is there a guiding principle (besides trial and error) that we may use to set this parameter's value?

No.

Will 128 work in general and transfer to other problem domains?

Probably not. Different problems present different challenges; not all problems are the same difficulty.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.