Questions tagged [convolution]
Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.
475
questions
1
vote
0
answers
9
views
Why does GAP at the end of FCN for MTSC work?
I have a binary MTSC (Multivariate Time Series Classification) problem where i train a CNN, namely a FCN (or Fully Convolutional Network) to predict class 0 or class 1 based on a multivariate time ...
- 11
1
vote
1
answer
32
views
Percentiles of a distribution of weighted summary statistics
Suppose I have a collection of different independent probability distributions, $\{ P_i(X)\}_{i=1}^N$, each with their own support $I_i$. I know that the $10^{th}$ percentile of a given distribution ...
- 117
0
votes
1
answer
78
views
What is the variance of convolution of two random variables?
Consider two random variables $Z$ and $W$. Given the variances of $Z$ and $W$, how can we compute the variance of their convolution $Z \circledast W $?
As an example, please consider the case of noise ...
0
votes
0
answers
25
views
PDF of difference of uniform distributions [duplicate]
Main questions are in bold but feel free to correct me if I'm wrong somewhere else. As far as possible, I need both intuition and formal explanation.
Let $X \sim Uniform(a,b)$ and $Y \sim Uniform(c,d)$...
24
votes
4
answers
2k
views
Probability that sum of binary variables is even
Let $S_i \in \{0,1\}$, $i=1,\dots,N$ be $N$ independent random binary variables, each taking the value 1 with probability $0 \le p_i \le 1$ (and the value 0 with probability $1-p_i$).
I am interested ...
- 4,420
1
vote
0
answers
24
views
What is the distribution of a RV with the constant random variable? [duplicate]
For random variables (rv) $X$ and $Y$ on a space $\Omega$:
Assume the rv $X\sim f_0$ distributed and $Y(t)=c$ is a constant rv, i.e. $Y\sim \delta(t-c)$ using the $\delta$-distribution as a short ...
- 209
0
votes
0
answers
47
views
Why is the maximum path length for convolutional layer $O(n/k)$ in attention is all you need paper?
In the table-1 third row it is being mentioned. Why is it $O(n/k)$? Take for example 1d convolution of 2 over 9 tokens with stride $1$. It won't be $n/k$ or $9/2=4.5$ rather it would be roughly $n-1$ ...
0
votes
0
answers
14
views
Inference on latent variable with observation of its convolution with itself
Problem
I have an inference problem where the data observed are univariate random numbers whose distribution is obtained as follows. A latent random variable X is first sampled from a parametric ...
0
votes
0
answers
43
views
Weighted Average of Uniformly Distributed RV [duplicate]
Let $x \sim U[0,1]$ and $y\sim U[0,1]$. Let $z= \omega\, x+ (1-\omega)\,y$, where $\omega\in[0,1]$. The pdf of $z$ is a trapezoidal distribution over $[0,1]$:
\begin{equation*}
\begin{aligned}
f(z)&...
1
vote
1
answer
71
views
Convolutional Neural Networks - Flattening with multiple feature maps
I have a very simple question about CNNs, which I unfortunately couldn't find an explanation for.
Imagine we have a CNN, that has four filters (eg right, left, top, bottom edges) each of those outputs ...
- 13
3
votes
1
answer
94
views
How to obtain the last convolutional layer of a model in torchvision for applying grad cam?
I'm using efficient net b0 from torchvision for training a classifier for cifar10. I would like to apply grad cam for generating saliency maps for explaining the predictions. However, I'm not sure ...
- 978
0
votes
0
answers
27
views
Empirical Mode Decomposition(EMD) + CNN for time series forecasting
I'm currently working on a time series project, and I intend to employ the EMD+CNN technique for forecasting the output. Upon applying EMD to the training data, I obtained a total of 14 Intrinsic Mode ...
- 1
0
votes
0
answers
24
views
CNN regression: output mimic the input test data
I am working on convolutional neural network regression problem using U-Net architecture for computing poisson equations. The input data is particle distribution (rho) and the output is electrical ...
0
votes
0
answers
20
views
Why do causal convolutions require positional encoding for image generation?
I'm following this course on deep learning. In lecture 10.2 (page 23 in the handout for the lecture) causal convolutions are introduced as a method for image generation.
A bit later it is said, that ...
- 23
0
votes
0
answers
22
views
Computing an integral that reduces to $\mathbb{P}[X>Y]$
Problem
Evaluate $$I=\int_{-\infty}^\infty \frac{e^{-\frac{1}{2}\left(\frac{x-\mu)}{\sigma} \right)^2}}{\sigma \sqrt{2 \pi}}\frac{1}{1+e^{-x}}\, \mathrm{d}x$$
My attempt
Now the first part of the ...
- 285
0
votes
0
answers
22
views
Exact Successor State Distribution for a Pendulum
I want to solve the following problem. Suppose we have a simple pendulum, which follows the differential equation
\begin{equation}
\dot{x} = f(x) = [x_2, -\sin(x_1)]^T, \text{with } x=[x_1, x_2]^T.
\...
- 307
2
votes
1
answer
32
views
No activation function between two convolutional layers in MUNIT?
I'm reading the code of NVIDIA's MUNIT, the code of the resnet is as follows:
...
- 21
1
vote
1
answer
128
views
Is it wrong to view convolution as template matching?
I am reading about the convolution operation but I can't see how it can be seen as template matching.
Suppose that we convolve the input $\mathbf{X}$:
$$
\begin{bmatrix}
1 & 0 & 0 & 0 \\
0 ...
- 471
2
votes
1
answer
173
views
Why convolving a function with a Gaussian kernel is the same as adding a Gaussian noise to the input? [duplicate]
I am implementing accelerated Langevin Dynamics (LD) for posterior estimation with prior presented with deep autoregressive network from paper [1]. I have a question about the prior smoothing ...
- 23
3
votes
1
answer
116
views
Probability Density of the Sum of Two Un-identical Uniform Random Variables
Let $X$ ~ Uniform$[a,b]$ and $Y$ ~ Uniform$[c,d],$ where $a\le b\le c\le d.$ Find the probability density of $Z = X + Y.$
I know I have to use the convolution formula
$$f_Z(z) = \int_{-\infty}^\...
0
votes
0
answers
23
views
Cannot apply simple OLS model in the case of low resolution devices resulting in Fourier space convolution
There's a problem which often comes up within my field and doesn't seem to be approachable analytically. Any suggestions or direction towards the class of problems this falls under would be helpful.
...
- 1
1
vote
1
answer
181
views
When is a conditional hazard rate increasing?
Cross posted from Mathoverflow
Let $X$ and $Y$ be two random variables such that $X\sim Exp(\lambda)$ and $Y$ have positive support and (strictly) increasing hazard rate $h_Y$. $X$ and $Y$ are ...
- 11
3
votes
2
answers
119
views
Convolution of two functions doesn't fit my data as I thought it would
I have simulated a Gaussian curve in 50 bins of data. I have then repeated this many times, drawing the amplitude of the Gaussian from a log-normal distribution. Here are a 10 realizations:
(IMAGE 1)
...
- 1,203
2
votes
0
answers
49
views
Estimating the distribution of a sum of two random variables if the family of one of the variables is known
Assume I have a random variable $Y=X_1+X_2$. I want to estimate the distribution $f$ of $Y$ given a sample $y_1,\ldots,y_N$. If this was all that is known about $Y$ the best way would probably be to ...
- 2,661
2
votes
0
answers
88
views
Calculating convolution in R [closed]
I am struggling to get the correct answer for the simple calculation of convolution in R.
The convolution of $f(t) = e^{-t}$ and $g(t) = \sin(t)$ is:
$$
(f * g)(t) = 1/2 \left( e^{-t} + \sin(t) - \cos(...
- 685
0
votes
1
answer
129
views
How to backpropagate transposed convolution?
I'm currently learning Convolutional Neural Networks and am stuck on trying to figure out how to compute gradients in a layer that uses transposed convolution. Also, how do I calculate the gradients ...
- 1
3
votes
4
answers
661
views
Probability of a given result with multiples of mixed dice with different number of faces
What is the formula to calculate the probability of getting 41 when I throw two 10-sided dice and four 8-sided dice? I’m looking for an algorithm for the general case of throwing multiples of two sets ...
- 275
0
votes
0
answers
7
views
Are the two formulas for computing the output shape of a convolution equivalent? Computing the floor before or after adding $1$?
I've come across two different formulas in my studies to calculate the output shape of a convolution.
Below, $I$ is the input image size, $K$ the filter size and $S$ the stride.
$$
\lfloor \frac{I - K ...
- 290
0
votes
0
answers
198
views
Replacing a fully connected layer with a 1x1 convolution vs with fxf convolution
Suppose an input of shape (width x height x channel_num) = (10 x 10 x 15) is obtained from previous convolutional layers, and this input is about to be inserted into a fully connected (fc) layer of K=...
- 37
3
votes
2
answers
136
views
Calculating the distribution of $X-Y$
One can find the distribution of $X+Y$ where $X$ and $Y$ are independent random variables using this formula
$$f_{X+Y}(a)=\int_{-\infty}^\infty f_X(a-y) f_Y(y) dy$$
I'm wondering how to adapt this ...
- 133
5
votes
1
answer
64
views
what are the differences between receptive field (RF) and field-of-views (FOV) in DeepLab papers?
I am learning the deeplab models. However, some concepts in the papers made me confused. Receptive field (RF) and field-of-views (FOV) are two concepts mentioned in the Deeplabv1 paper. I know that ...
- 311
9
votes
3
answers
375
views
Density of $|t_1 - t_2|$ where $t_1$ and $t_2$ are iid with $P(t) = \alpha e^{-t\alpha}$
I am trying to answer the following question from my quantum mechanics textbook and my probability theory is admittedly rusty (this is not schoolwork as should be clear from my post history on Phys ...
- 203
1
vote
0
answers
67
views
Order Statistics - Percentile Range of Normal Mixture of Normals
Say I have draw N values from a normal distribution [$\mu_1$, $\sigma_1$]. Below are 10 sampled points compared to the normal distribution they're sampled from
I then create a normal mixture of ...
- 11
4
votes
0
answers
114
views
Distribution closed under convolution and truncation followed by convolution
Let $D(\theta)$ denote an absolutely continuous distribution on $\mathbb{R}$. (The finite dimensional vector $\theta$ collects the parameters of the distribution.) Assume that the p.d.f. of $D(\theta)$...
- 535
1
vote
1
answer
153
views
How can I build a 6D Convolution Layer in a CNN, using R or Python? [closed]
As stated in the title, I would like to build an N-dimensional Convolutional Layer as part of a Convolution Neural Network, without doing dimensionality reduction on my data; because I have multiple ...
- 111
0
votes
0
answers
26
views
Unclear Architecture of MNIST Neural Network
I am trying to reproduce a Neural Network trained to detect whether there is a 0-3 digit in an image with another confounding image. The paper I am following lists the corresponding architecture:
A ...
- 33
1
vote
1
answer
75
views
What "Convolution filters along the time axis" means?
Suppose that I have a tensor of height:25 and width:50. Height is my temporal axis, therefore I have a window of 25 time steps. Therefore my input tensor is:
I want to extract temporal features / ...
- 203
1
vote
0
answers
41
views
Relationship between convolutions in neural nets and probabilistic convolution
In probability theory, convolution extends to produce the distribution of the sum of two independent random variables. I've only ever seen this in the context of univariate random variables.
However, ...
- 309
1
vote
0
answers
14
views
Are Class Activated Maps an example of backpropagated Explainable AI?
Class Activated Maps (NOT GradCAM) is listed in van der Velden et al as a back-propagation approach in table 1, but that is not actually true, is it?
CAM doesn't use back-propagation in the method, ...
- 33
1
vote
0
answers
23
views
How to calculate how much each error source contributes to an overall accuracy metric?
I am building a model of overall accuracy for a robotic system, I have various error sources from assembly, calibrations, measurements, imaging, for each error I have a PDF of the error it induces on ...
- 11
0
votes
2
answers
95
views
How to average multiple non-normal distributions?
I have the following statistics of two independent random variables:
First random variable:
...
- 49
1
vote
0
answers
40
views
Mean conditional on sum
Let $X$ and $Y$ be two random variables. I am interested in how $\mathrm{E}[X | X + Y = z]$ changes as $z$ changes.
Intuitively, if $X$ and $Y$ are independent, the conditional mean should be ...
- 11
4
votes
3
answers
3k
views
How to determine the no of multiplication operations in convolution operation?
Let's say We have an input of size 28×28×192. We apply 32, 5×5 convolution filters with padding "same". How many multiplication operations will be there in total?
I know there will be ...
0
votes
0
answers
65
views
How to take 1000 samples from distribution X and then use MLE to prove they came from distribution X?
I am trying to do:
find 1000 points that represent samples from distribution X with parameters $(a,b,c,\ldots, d)$
be guaranteed that the MLEs for those 1000 points are $(a,b,c,\ldots, d)$ with $99\%$...
- 127
0
votes
0
answers
12
views
What is difference between the joint probability distribution and the sum/convolution, of 2 dists? [duplicate]
Google is coming up a bit short when I searched for "joint vs sum random variables".
Perhaps someone can provide an authoritative answer to compare and contrast the sum/convolution of 2 ...
- 127
5
votes
2
answers
2k
views
Why does a 1D convolution increase the size of the output, while a 2D convolution tends to decrease (such as in a CNN?)
The function np.convolve is a 1D convolution (e.g. when both inputs are 1D). It results in a larger output size.
...
- 255
0
votes
1
answer
82
views
Convolution of two multivariate guassian distribution for the posterior predictive distribution
To find the full conditional distribution of $\eta$ for a Gibbs sampling algorithm , I have to show that
$$
p(\eta|-) \propto \int N(\eta;\Omega(\Lambda+\Delta^{-1}\mu),\Omega) N(\mu;\hat{\mu},\Delta/\...
- 1
3
votes
0
answers
128
views
How to incorporate prior knowledge into a CNN?
I'm pretty new to Bayesian inference and machine learning, so I think I'm just lacking the right words to search for a paper that addresses this topic, so here goes:
I'm trying to do image ...
- 131
0
votes
1
answer
109
views
What kind of architecture to use for non-binary output multi-label image clasification
I want to make a network for making multi-label attribute classifications on images of clothing.
This is a simplified case of what I want to do, I have 9 different attribute categories that I wish to ...
- 101
0
votes
0
answers
34
views
Why does my network not learn a single image perfectly?
I have a convolutional neural network that uses Resnet(18,34 or 50 doesn't matter) as the backbone and pretrained weights from ImageNet.When I try training it with a single image for 50 or so epochs, ...
- 1