All Questions
Tagged with neural-networks probability
22 questions with no upvoted or accepted answers
3
votes
0
answers
122
views
Correlated random variables and ensembles (law of large numbers?)
Consider $n$ i.i.d random variables. By the law of large numbers (LLN) the sample average would converge after some time to the expected value.
Let's assume the random variables are correlated. Would ...
- 538
2
votes
0
answers
72
views
How does Dempster-Shafer Theory of Evidence relate to Deep Learning?
I am reading this article and it has the following phrase - "Dempster-Shafer Theory of Evidence assigns belief masses a set of classes (unlike assigning a probability to a single class)". ...
- 644
2
votes
0
answers
118
views
How to prove that neural network estimates posterior distribution
Let's say that I train a neural network in a classic binary classification setting where all the training data has labels in $\{-1, +1\}$. From my understanding, if I train the network with a log-loss ...
- 53
2
votes
0
answers
35
views
Comparing prediction distributions across many classes
I have a deep neural network with a Softmax classifier as the final layer. For each observation, the network produces a probability distribution over the 64 possible classes that the observation can ...
- 5,146
1
vote
0
answers
38
views
Neural Networks Miscalibration Measure
I have read these two papers related to the neural network miscalibration problem: "On Calibration of Modern Neural Networks" and "Multivariate Confidence Calibration for Object ...
- 11
1
vote
0
answers
17
views
When imputing missing labels Y1 == NaN during training, how do additional target vectors (Y2 != NaN) impact learning Y1==NaN?
I am training a Mixture Density Network (MDN) to map from continuous input vectors X to continuous targets Y [i.e. X -> Y]. There are missing labels on vector Yi, which I impute from the mdn (as ...
- 11
1
vote
0
answers
165
views
How to understand the density in machine learning?
We can calculate the conditional density using Eq.1[3].
$$
p_{\theta, \Lambda}(y \mid \boldsymbol{x})=\frac{\exp \left(f_{\theta, \Lambda}(\boldsymbol{x})[y]\right)}{\sum_{k=1}^{n} \exp \left(f_{\...
- 11
1
vote
0
answers
258
views
How to sample from a distribution approximated by a Neural Network?
There are a few models already that approximate distributions with a neural network i.e.: energy models define a density function $f(x)= e^{S(x,w)}/Z$ where $S$ is a neural network and $Z$ is a ...
- 768
1
vote
0
answers
66
views
Skipgram model theory confusion
In the output layer of a skipgram model, there are $|\text{Context}|*|\text{Vocab}|$ values. And for each context word, the values are basically the dot product of the input word representation and ...
- 133
1
vote
0
answers
264
views
Softmax Classifier gives weird confusion matrix
I'm currently working on a problem of binary classification in keras and have decided to use the softmax function as the activation function for my final classification layer. My current network is as ...
1
vote
0
answers
642
views
KL divergence of Variational Autoencoder not decreasing
I have been trying to train a VAE to generate cat pictures. The images are of size 64 by 64. But upon training my reconstruction loss decreases whereas my KL loss remains constant/slightly increases. ...
- 11
1
vote
1
answer
548
views
Probability that a population mean exceeds a threshold
In Googling this question, I see that there are a variety of similar tests but I couldn't find anything given the exact way I'm approaching this problem. This might be something obvious but I'm not ...
- 173
1
vote
0
answers
414
views
Did I understand the usage of Gumbel-Softmax reparametrization correctly?
I am working on a deep learning model, which has a mixture of experts formulation like $\log p(y|x)=\log \sum_{z}p(y|z,x,\theta)p(z|x,\phi)$. So, each $p(y|z,x,\theta)$ is a deep learning classifier, ...
- 2,310
1
vote
0
answers
35
views
Normalization: different approach
Today I had a discussion about the right way to normalize data, especially image data. The standard approach, as found in many tutorials etc., seems to be following. For example, the data has a range ...
- 11
1
vote
0
answers
734
views
Splitting Probability Distributions Into Many Factors : The Deep Learning Book
In chapter 3, section 3.14: Structured Probabilistic Models in The Deep Learning Book the authors write this, with the following equation.
Machine learning algorithms often involve probability ...
- 11
1
vote
0
answers
704
views
How can an ensemble perform worse than all but one of its constituents?
I came across a very unusual situation:
I trained 5 deep nets on a problem.
4 of the 5 had excellent in- and out-of-sample accuracy.
I trained a classifier on the probability outputs of the 5 deep ...
- 743
1
vote
0
answers
54
views
Can we obtain probability distribution of a repeatable event using Neural Networks?
We have a data where input of every sample corresponds to how many times a dice is rolled. The output is the sum of all the outcomes.
For instance the data is ...
- 111
1
vote
0
answers
330
views
Hammersley–Clifford theorem
I'm reading this paper http://image.diku.dk/igel/paper/AItRBM-proof.pdf and I got stuck in page 4 with equation (1) that's based on Hammersley–Clifford theorem. I'm not good in reading set theory ...
- 8,411
0
votes
0
answers
87
views
Normalizing Flow Penalization
I am looking to train a normalizing flow, specifically a Masked Autoregressive Flow model. However, this model leads to high variance on lower dimensional, less complex data. I am using a neural ...
- 19
0
votes
0
answers
62
views
Understanding deterministic models via probabilistic graphical model
I have read a few tutorials how we can think of deterministic neural networks with the help of probabilistic graphical models. Very often they would offer an image as seen bellow and say, our model is ...
- 261
0
votes
0
answers
26
views
How to name a machine learning algorithm where we used very small amount of labeled data?
I have to describe a machine-learning algorithm that needs only very small dataset. I cannot say it is unsupervised because I already used the labels in the training. Also, I cannot say semi-...
- 159
0
votes
0
answers
108
views
Joint probability distribution of correlated data points
I have a query with respect to joint distributions.
Here, each output data point in $\mathbf{y}$ is conditionally independent given the inputs $\mathbf{x}$ and the mapping $f:\mathbf{x}\rightarrow \...
- 31