3
$\begingroup$

Neural Networks have "temperature" - how much randomness should be added to the result.

Would it be possible to use that temperature to get not just single number prediction, but the probability distribution of the predicted value?

Like - you need to predict price of microsoft stock tomorrow, you increased temperature, ran prediction 1000 times, get 1000 slightly different predictions, and build the histogram.

I'm sure that would work, but would it be more or less real distribution or just some funny similarly looking random artefact?

Also, would that temperature thing work with CNN - Convolutional Neural Networks?

Improve this question
$\endgroup$
2
  • $\begingroup$ Whether this matches some real distribution or not depends on what that distribution is. What is 'real' is exactly one value, actual the stock price tomorrow, not a distribution over multiple values. So, to evaluate the distribution received from your network, you need some probabilistic model of the stock prices to compare it with. $\endgroup$
    – Discrete lizard
    Commented May 22, 2019 at 6:20
  • $\begingroup$ @Discretelizard thanks, I flagged it for migration. As for the "real distribution" - I meant more or less close to real, like with Markov Chain - it would allow to get something close to real distribution, I wonder if that's similar or not. As for the "real" - of course nobody know the ground truth. $\endgroup$
    – Alex Craft
    Commented May 22, 2019 at 13:26

1 Answer 1

Reset to default
1
$\begingroup$

No. I would also not characterize the temperature as "how much randomness should be added to the result."

The concept of temperature comes from Boltzmann's distribution, which is linked (or serves as inspiration) to the softmax function, that is applied on activation outputs. The physics analogy is heating up the gas, and observing how the distribution of molecule speeds change as a result. I would say that with temperature the entropy increases, so randomness does increase, making it more likely for state transitions.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, do you know if there's other way to get that distribution? $\endgroup$
    – Alex Craft
    Commented May 22, 2019 at 14:21
  • 1
    $\begingroup$ You mean probability distribution? $\endgroup$
    – Aksakal
    Commented May 22, 2019 at 14:25
  • $\begingroup$ yes, how to predict the probability distribution with the neural network.. $\endgroup$
    – Alex Craft
    Commented May 22, 2019 at 14:45
  • $\begingroup$ @AlexeyPetrushin, it can't be done, I'm afraid, in general case. Unless you build a network which is specifically built to predict probability distributions, i.e. it's output is a distribution e.g. its quantiles $\endgroup$
    – Aksakal
    Commented May 22, 2019 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.