All Questions
Tagged with neural-networks probability
77 questions
1
vote
2
answers
354
views
Probabilistic model of neural network
I read Neil's presentation and found this joint model is confusing:
$$p(y_*|y, X, x_*) = \int p(y_*|x_*, W)p(W|y, X)dW $$
where $W$ contains $W_1$ and $W_2$ and $p(W|y,X)$ is posterior density
Can ...
- 547
0
votes
1
answer
579
views
using DNN to find out the pdf of a regression problem
When we use deep neural networks (DNNs) to solve a 1-dimention regression problem, we can approximate data distribution with the output of a DNN like the picture below.
My question is that DNN does ...
- 135
1
vote
0
answers
734
views
Splitting Probability Distributions Into Many Factors : The Deep Learning Book
In chapter 3, section 3.14: Structured Probabilistic Models in The Deep Learning Book the authors write this, with the following equation.
Machine learning algorithms often involve probability ...
- 11
1
vote
0
answers
704
views
How can an ensemble perform worse than all but one of its constituents?
I came across a very unusual situation:
I trained 5 deep nets on a problem.
4 of the 5 had excellent in- and out-of-sample accuracy.
I trained a classifier on the probability outputs of the 5 deep ...
- 743
53
votes
6
answers
43k
views
Why is softmax output not a good uncertainty measure for Deep Learning models?
I've been working with Convolutional Neural Networks (CNNs) for some time now, mostly on image data for semantic segmentation/instance segmentation. I've often visualized the softmax of the network ...
- 659
1
vote
1
answer
966
views
Conditional probability vs. likelihood - neural networks
In Goodfellow et al.'s Deep Learning, the authors write about recurrent neural networks on page 371:
The total loss for a given sequence of $\mathbf{x}$ values paired with a sequence of $\mathbf{y}$...
- 3,263
2
votes
1
answer
1k
views
Normalizing probability of sequence by its length [closed]
Is there any commonly accepted method to derive probabilities of sequences that are not dependent on length?
Background:
I'm trying to generate sequences of symbols from the individual probabilities ...
- 171
5
votes
2
answers
2k
views
Is graduate level probability theory (Durett) used often in ML, DL research?
I am interested in machine learning; I have a particular liking for RNNs. I have coursework in some areas of computer science, e.g., data mining, optimization for ML algorithms, deep learning, and an ...
- 5,800
8
votes
1
answer
2k
views
Why deep learning prefer the probability distribution with a sharp point?
I am reading Ian Goodfellow's book about deep learning and when it introduces exponential distribution, it says "In the context of deep learning, we often want to have a probability distribution with ...
- 201
1
vote
1
answer
171
views
Is verification with test data sufficient to rule out overfitting of neural network?
I have a dataset of N normalized features, and outcomes of the form 1.0 and 0.0 (win and loss), split 50/50 into training and test data (about 50000 samples each).
I train the artificial neural ...
- 219
4
votes
2
answers
8k
views
Softmax Multiclass Classification
How do we associate a class to every output unit in a multilayer neural network architecture? I mean we assign the output to the class with maximum probability, but how do we decide which neuron ...
- 211
10
votes
4
answers
25k
views
Neural networks output probability estimates?
Suppose my training data contains ~100 variables, and each example is tagged as "success" or "failure".
I understand how a neural network can be used to try and predict success vs failure based on ...
- 219
8
votes
1
answer
9k
views
The effect of temperature in temperature sampling
I was reading this while I found:
The high temperature sample displays greater linguistic variety, but
the low temperature sample is more grammatically correct. Such is the
world of temperature ...
- 1,395
-1
votes
1
answer
123
views
Regularity of functions approximated with neural networks
Are there any papers pertaining to smoothness / regularity of functions that are approximated with artificial neural network?
2
votes
0
answers
35
views
Comparing prediction distributions across many classes
I have a deep neural network with a Softmax classifier as the final layer. For each observation, the network produces a probability distribution over the 64 possible classes that the observation can ...
- 5,146
0
votes
2
answers
2k
views
Getting probability from Restricted Boltzmann Machine
Let's consider a trained Restricted Boltzmann Machine model. It was trained to maximize P(v). Since it's a generative model, how can I get a probability of an input vector which it is supposed to ...
- 21
1
vote
0
answers
54
views
Can we obtain probability distribution of a repeatable event using Neural Networks?
We have a data where input of every sample corresponds to how many times a dice is rolled. The output is the sum of all the outcomes.
For instance the data is ...
- 111
11
votes
2
answers
11k
views
How to compute bits per character (BPC)?
In one of Alex Graves' papers (and several other authors as well) utilize the term bits per character (BPC). The paper that I am referencing here is "Generating Sequences with Recurrent Neural ...
- 213
4
votes
2
answers
1k
views
How to evaluate the quality of the probability distribution output of a classifier?
In a classification problem, I have trained a neural network which outputs class probabilities for a given input. For a new input, I now want to evaluate the "quality" of the neural network's ...
- 7,129
10
votes
2
answers
4k
views
How is softmax unit derived and what is the implication?
I'm trying to understand why the softmax function is defined as such:
$\frac{e^{z_{j}}} {\Sigma^{K}_{k=1}{e^{z_{k}}}} = \sigma(z)$
I understand how this normalizes the data and properly maps to some ...
- 247
0
votes
1
answer
205
views
What is obtained from the product of a probability and a log probability ratio? [closed]
I'm looking at the commonly used artificial neural network model that has nodes and connections.
Quick refresher
A connection has a source and target node, and a weight. The output of the source ...
- 878
1
vote
1
answer
125
views
Composition of bankruptcy probability and firm size
I'm using neural network for a binary classification problem of bankruptcy prediction using patternnet function in MATLAB, so i ...
- 4,271
1
vote
0
answers
330
views
Hammersley–Clifford theorem
I'm reading this paper http://image.diku.dk/igel/paper/AItRBM-proof.pdf and I got stuck in page 4 with equation (1) that's based on Hammersley–Clifford theorem. I'm not good in reading set theory ...
- 8,411
7
votes
1
answer
3k
views
Combine several softmax output probabilities
I would like to combine the outputs of five neural networks, each with a softmax output layer of three classes each. A typical, example output is shown below:-
where Figure 1 is the output of model 1,...
- 4,869
7
votes
4
answers
8k
views
Methods & CRAN packages to predict probability using neural networks or others machine learning algorithms
I have a medical database containing 7 input variables (4 are binary) and a binary outcome variable (Survival: yes/no). My objective is to train and test an algorithm that predict probability of ...
- 75