There are several different choices available to you. What is "optimal" depends on the specific application, both in terms of your own needs & convenience (which YOU get to choose), and in terms of the quality of response of the Neural Network, Machine Learning algorithm, or whatever it is that you are trying to train. In many cases you will only find out what works best by trying the different alternatives.
Some possibilities are:
1) 12 different binary variables, e.g. JAN = 1 if actual month = January, otherwise 0; similarly for each succeeding month, e.g. FEB = 1 or 0, etc.
2) A single variable with 12 discrete values {Month = 1, 2, ..., 12} or equivalent. This might be OK if you really want to treat each month separately, but ...
3) If you are looking at forecasting temperature or rainfall for example, the weather itself really doesn't really care if it is 30th June or 1st July, so you might be better off using a decimal fraction of the year which then becomes a continuous variable changing day-by-day.
4) As you correctly point out, there is a potential problem of modulo distance, i.e. if you use numbers 1, ... , 12 then each month differs by 1 from the previous month, except for Dec=12 & Jan=1, so you may need to use something like: IF MONTH - PREVIOUS MONTH < 0 THEN MONTH DIFFERENCE = MONTH DIFFERENCE + 12.