Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
435 views

What is the Identity of a convolution layer in a Neural Network?

I wanted to know what the identity of a convolutional layer of a neural network was. For standard convolution operation in mathematics the identity is the delta function, however, convolutions in ...
1
vote
0answers
178 views

How to add bias in convolution transpose?

My question is regarding the transposed convolution operation (also commonly called deconvolution or upconvolution). In TensorFlow, for instance, I refer to this layer. My question is, how / when do ...
2
votes
1answer
914 views

Padding and stride in backpropagation of a conv net

I am trying to implement the back-propagation of a simple convolutional network. Specifically I understand that one of the steps is the convolution of the gradients coming from the next layer, with ...
10
votes
2answers
2k views

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to ...
1
vote
0answers
52 views

When is the convolution of symmetric bimodal densities unimodal?

Let $X$ and $Y$ be real valued random variables with densities $f_X$ and $f_Y$. It is well known that if $f_X$ and $f_Y$ are symmetric about zero and unimodal then their convolution $f_X \ast f_Y$ is ...
1
vote
0answers
50 views

The Application of Convolution and Delta Functions

$$K=Z-(X+Y)$$ $Z$ is any discrete value. I applied convolution for PDF of two exponential random variables: $X$ and $Y$. Which is like that here. Now I need to compute PDF and expected value of $K$? ...
0
votes
0answers
116 views

Solve a linear equation system of convolutions

For linear systems of equations, like Ax = b, the solution that minimizes the mean squared error and the norm is given as ...
1
vote
0answers
199 views

What is the differennce between invariance to translation, covariance to translation and equivariance to translation?

I get stuck at understanding the difference between invariance to translation, covariance to translation and equivariance to translation in the context of of convolutional neural network. What does ...
5
votes
0answers
112 views

Finite sum of beta prime iid random variables

The beta prime distribution is infinitely divisible, as proved in Steutel and van Harn, 2003 (Appendix B). Sadly, in this book, there is no espression of the parameters of the distribution of n ...
1
vote
1answer
818 views

Concatentation of feature maps in U-net

I am looking at the following snippet of code: ...
1
vote
1answer
50 views

What does the matrix $M = [diag(m_{:,1}),\ldots,diag(m_{:,m})]$ look like?

I'm reading this paper about a convolutional neural network (CNN) to model sentences. I think I understand the paper reasonably well until section 3.4. Please consider the following text taken from ...
1
vote
0answers
91 views

How to insert feature vectors as additional channels in conditional DCGANs

I understand that in a fully-connected GAN you can simply concatenate the flattened image and feature vector as input for the network. For convolutional GANs I've read that you should add the feature ...
6
votes
3answers
2k views

Sum of normal independent random variables with coefficients

I'm trying to wrap my head around linear transformations to random variables (with coefficients > 1). Consider the two random and independent variables $X$ and $Y$ where: $$X \sim \mathcal{N}(0,1)\...
1
vote
1answer
136 views

convolution and deconvolution of random variables of different dimensions

Preliminary: Let's say we have $Y=X+Z$ ($Y$ is data, $X$ is latent variable and $Z$ is noise), where the random variables are all in $\mathbb{R}$. Then an inverse Fourier transform leads to \begin{...
0
votes
0answers
132 views

How to create a wave-net-like CNN?

I would like to create a CNN in a similar way to WaveNet architecture. i.e. on the first layer it takes convolves 3x3 areas. In the next layer it also convolves 9 pixels but spaced out like this: <...
4
votes
0answers
60 views

Multivariate stable distribution

I know that if $\pmb{X}_1$ and $\pmb{X}_2$ are independent copies of a $n \times 1$ random vector $\pmb{X}$, then $\pmb{X}$ is said to be sum stable in $\mathbb{R}^n$ if $a\pmb{X}_1 + b\pmb{X}_2 \...
4
votes
2answers
357 views

Prove that the mean value of a convolution is the sum of the mean values of its individual parts

Prove that the mean value of a density function convolution, if the mean values exist (they may not for fat-tailed distributions) is the sum of the mean values of the density functions used to make ...
3
votes
1answer
1k views

Kernel sizes for multiple convolutional layer neural networks

In most examples I've seen of CNNs with multiple two or more convolutional layers the second layer has more kernels (feature masks) than the second layer, usually around twice as many. My intuition is ...
1
vote
2answers
155 views

Survival in two period game: mean of z|z<v with z=xy, x~U(a,b) and y~U(c,d)

I am looking for the functional form to describe the following: A random shock $x\sim Uniform(a,b)$ is multiplied with a second shock $y\sim Uniform(c,d)$. What is the mean value of all combined ...
2
votes
0answers
96 views

Flip Weights. How to change bias?

I am reading serialized weights and put them into a tensorflow network: tf.tanh(tf.nn.conv2d(t_im0, weights, strides=strides0, 'SAME') + bias) If I flip the ...
0
votes
0answers
77 views

How to know the shape of the future feature maps knowing the shape of the kernel applied in an image?

Let's say that I have a [32*32] image and i want to convolve a 5*5 filter on it. The result of this convolution will be a 28*28 image. But how do we know that it will give a 28*28 image. Is there a ...
1
vote
0answers
56 views

Short Sentence Generation using CNNs

I am investigating whether building a classifier for sentence classification using CNN can be used for sentence generation. Say, we are classifying news articles' titles (classes such as sports, ...
0
votes
1answer
33 views

Integration Limits on Functions of Two Random Variables

I am working on the following problem and am struggling with how to come up with the particular limits of integration. I've done some maths voodoo to get part of the same argument as the solution, but ...
0
votes
1answer
266 views

Multiple output of neural network

I am trying to wrap my head around multiple output of neural networks, especially output of CNN in image classification with localization: Lets say we have CNN with 2 cond layers (conv + pool, conv + ...
3
votes
1answer
1k views

CDF of Z=XY with X~Uniform(0.5,1.5) and Y~Uniform(0.8,1.5)

I am looking for the CDF of the product of two independent random variables (X and Y) with uniform distributions. Both random variables uniform distributions have interval boundaries (upper and lower ...
34
votes
10answers
36k views

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution, whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$; the arithmetic sum ...
2
votes
1answer
4k views

Dilation Rate in Convolution Neural Network

What is the dilation rate in dilated convolution as mentioned in the paper here.
4
votes
1answer
499 views

What is the convolution of a normal distribution with a gamma distribution?

Is there a closed form expression for the convolution of a normal distribution (ND) with a gamma distribution (GD)? There does not seem to be a direct method of solving this convolution.
2
votes
1answer
67 views

What is the spectral domain?

From https://arxiv.org/pdf/1611.08097.pdf, "Geometric deep learning: going beyond Euclidean data". Here are some uses: "methods of signal processing on graphs, which have previously been reviewed in ...
0
votes
1answer
4k views

Feature Maps vs Channels in CNN

What's the difference between feature maps and channels in Convolutional Neural Network. I understand that number feature maps is determined by number of filters. I'm thinking that both of these ...
1
vote
1answer
1k views

Why are nonlinear activations used in convolutional neural networks?

Isn't convolution non-linear already? Do we need a nonlinearity in a different layer as as well? I understand the reasons why ReLU is good cited in this answer, but is there an intuitive reason we ...
3
votes
1answer
1k views

Why building the sum of a filter over several channels in a convolutional layer?

Let's say I have RGB input data (3 channels) and a convolutional layer which has just one filter with a depth of 3. The output data will have a depth of 1 if we build the sum over the results of the ...
1
vote
1answer
133 views

How to change parameters of a convolution to get halve the former output size? [closed]

I have an existing CNN architecture which gets an image (size: $640 × 352$) and returns and image of the same size ($640 × 352$, via convolution and deconvolution). And I'm trying to halve the output ...
0
votes
1answer
39 views

Mismatch between analytical and simulation way for Bayesian estimation of binomial data

In short, Below, I'm asking why the analytical way of getting the posterior for the difference between two proportions (binomial data) and simulation way of the same don't match? (Note: All code is <...
1
vote
1answer
3k views

How does a 1-dimensional convolution layer feed into a max pooling layer neural network?

I'm having some trouble mentally visualizing how a 1-dimensional convolutional layer feeds into a max pooling layer. I'm using Python 3.6.3 and ...
1
vote
0answers
752 views

Convolutional neural network: why is my training accuracy suddenly dropping?

I am trying to understand this behaviour in my shallow CNN (only one layer followed by a sigmoid classification layer. I am training on about 2000 images per class. My learning rate was 0.01 and I was ...
0
votes
0answers
41 views

Neural net that preserves spatial information in output

I'm attempting to use a neural network as a kind of interpolator for a high-dimensional function. We're doing this to circumvent the need for a physical model that calculates this function exactly, ...
7
votes
2answers
13k views

Difference between Conv and FC layers?

What is the difference between conv layers and FC layers? Why cannot I use conv layers instead of FC layers?
2
votes
0answers
342 views

How to define a loss function for discrete fourier series?

In each batch there are 8000 sample points, and I apply discrete Fourier transform on them. The original samples are real valued, so only the half of the result is needed. The end result is 4000 ...
0
votes
0answers
56 views

Conditional PDF of Dependent RVs

So I'm given that $x, y$ are distributed as: $ P(\alpha) = \begin{cases} 1 & 0 \leq \alpha \leq 1 \\ 0 & otherwise \end{cases} $ First I needed to calculate $P_{S}(s)$ where $S =...
5
votes
1answer
693 views

Intuition for why sum of gaussian RVs is different from gaussian mixture

I know that in the case of Gaussian mixture, the "intuition" is that you're drawing from a PDF which itself is just a sum of weighted Gaussian PDFs. I don't understand the intuition behind how the ...
1
vote
1answer
165 views

Training a Convolution Neural Network with Statistical Data (Features Extracted from the Medical Image)

I'm currently working on a project to classify the medical images as normal and abnormal using convolution neural network with input as feature data (Statistical feature values extracted from medical ...
4
votes
1answer
4k views

What should regularization loss look like?

New to ML and Deep Learning. I am trying my hands on some image classification using resnet50. Here are some of the graphs I see on tensorboard: As you can see, my total loss is mostly being ...
1
vote
0answers
253 views

How do I model a pixel-wise regression convnet? What kind of loss should I use?

Given an input of (HxWxD), I want to output a confidence map of size (HxW) where each value is a probability. I'll try to be concise. I have 2 inputs: input_image of size (HxWx3) input_map of size (...
1
vote
1answer
2k views

Showing prediction output in Keras [closed]

I am doing a image classification with CNN using Keras. The training process was so far so good. But when it comes to the prediction, I couldn't figure out to extract correct prediction results out of ...
1
vote
0answers
206 views

sum product algorithm and Convolution Neural Network

I'm trying to understand the sum-product algorithm implemented using Convolution Neural Network by the paper [1,2] to solve the problem of human pose estimation. Human pose estimation is formulated ...
1
vote
0answers
662 views

convolution of two probability densities in MatLab

I am considering two exponential probability distribution functions with mean equal to 5 and 3. pdf1=@(x)exppdf(x,5); pdf2=@(x)exppdf(x,3); I want to compute the ...
4
votes
1answer
5k views

Batch normalisation at the end of each layer and not the input?

I am currently studying the paper of network implementation RCNN. The core module inside RCNN is the Recurrent Convolutional Layer (RCL), whose state evolves over discrete time steps. The ...
2
votes
1answer
712 views

Convolution of two binomial distribution

The problem and some of my thought are as followed, could you help check if I'm wrong. Suppose $X∼Bin(n_1,1/2)$ and $Z∼Bin(n_2,p)$, $0<p<1$ being an unknown parameter; $X$ and $Z$ are assumed ...
2
votes
0answers
206 views

How to retrain a model (Inception) with 'prioritised' images in certain classifications

I am new to machine learning, and have constructed a basic CNN classifier by retraining the last layer of the Inception v3 model with my own image set into two classifications. I did this in Python ...

1 2 3 4 5 7