I came across SELU, and I was wondering how to derive $alpha$ and $lambda$ in the function, because I wasn't able to understand the math in the reseach article. I did, however, find this website that drew the following values from the paper: $$alpha = 1.673263...$$ $$lambda = 1.050701...$$ But, now I'm slightly confused because I thought self-normalizing meant the function would adjust its $alpha$ and $lambda$ values over time with the influx of training data. Or have I misread something? If I could provide a link to a video or a website or if I could manage an explanation of the functions which define the alpha and lambda values I'd really appreciate it.
I don't claim to understand the full proof provided in the paper. However, I do know that $\alpha$ and $\lambda$ are constants, not learned.
I thought self-normalizing meant the function would adjust its alpha and lambda values over time with the influx of training data.
No, the term self-normalizing refers to the fact that using this activation causes the distribution of activations to roughly adhere to the standard normal distribution.