Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.

Filter by
Sorted by
Tagged with
0
votes
1answer
690 views

CNN convolutional layer backpropagation formulas

I tried to implement a CNN in Java but I am stuck at updating the weights in my convolution layer. I tried to create the following image that shows how I calculate each weight delta and error signals:...
5
votes
1answer
3k views

CNN vs RNN for time series classification

I am new to neural networks and after some research i read about CNN and RNN neural networks. The data that i am having is multiple different time series of numbers. So for example instead of input 1 ...
3
votes
0answers
148 views

Is this normal convolution or something special?

I am currently studying this paper (page 53) (mirror), in which the suggest convolution to be done in a special manner. This is the formula: \begin{equation} \tag{1}\label{1} q_{j,m} = \sigma \left(...
1
vote
0answers
67 views

What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?

In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
2
votes
3answers
656 views

Class Weight doesn't solve imbalanced dataset problem

I'm training convolutional neural network on imbalanced dataset, which has 9 classes. Number of classes in order is, 3000-500-500- ..... goes like this. Of course I'm not waiting %100 accuracy, but ...
1
vote
0answers
33 views

Do I model my training set on real world classification percentages?

NOTE: This is just a toy project - I'm trying to learn by doing. I am aware of the previous work in this area. This question is about how to correctly populate my training, validation and test sets in ...
1
vote
1answer
330 views

Are there simple networks where a ReLu between convolutional layers has significant value?

At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
0
votes
1answer
95 views

How to compute more efficiently in R the probability distribution of the sum of non-independent discrete random variables

I hope you are well. Let $\{s_0,\,s_1,\ldots,\,s_T\}$ be a sequence of discrete random variables and denote $S_t=s_0+s_1+\cdots+s_t$, with $S_0=0$. For all $t\in\{1,\ldots,\,T\}$, suppose that $s_t|\{...
2
votes
1answer
579 views

How many times should one train a convolutional neural network?

ML engineers usually train 50-100 times a network and take the best model among those. I am wondering how many times a CNN should be trained as training a CNN is costly and time-consuming too.
1
vote
0answers
414 views

Text classification: Lost in the world of Deep learning, NB, LSTM and CNN

Imagine I have a large text document, I want to be able to highlight some parts of the document if they belong to a category I've specified. I'm trying to figure out what's the best machine learning ...
1
vote
0answers
111 views

Applying 2D convolution to multichannel image

I have a single-channel image and want to make it multichannel by corrupting it with various sources of noise. Let's say the end dimension is [height , width , number_of_corrupted_copies]. How can I ...
1
vote
0answers
385 views

Sparse Pattern Recognition Using Convolutional Neural Networks

Context of Problem I am trying to troubleshoot communication issues between process sensors of a large manufacturing company. I have been given 1-year worth of data for 1,000 distinct processes (...
6
votes
2answers
2k views

Convolutional neural networks backpropagation

My question is regarding the answer to this question: Training a convolution neural network It seems like the answer is saying to change all the weights in a given filter by the same amount in the ...
0
votes
1answer
79 views

How to use convolution of 2 loss distributions?

My project is to set a distribution loss of PNL in CHF into a distribution loss in USD. To do this I will need to have a distribution loss of the spot rate CHF/USD. I have simulated this distribution. ...
1
vote
0answers
180 views

Distribution of sum of squares of elements of multinomial random vector

I am interested in the distribution of the sum of squared elements of a multinomial distribution. Specifically if: $$\vec X \sim \text{multinomial}(N,\vec p)$$ What is the distribution of: $$\sum_{i=...
6
votes
2answers
298 views

Convolution of random variables: unimodality of the likelihood function

Let $X_1, X_2,...X_k$ be random independent variables, each $X_i$ drawn from a Geometric distribution $\mathcal{G}(p_i)$, and let its convolution, or sum, be $Y = \sum_{i=1}^k X_i$. The likelihood ...
4
votes
2answers
519 views

Unlearning Neural Network? Prevent learning from a specific feature

Is it possible to train a NN to avoid the features that a different neural network finds? For example, let's train a simple 1 layer CNN with 1x1 kernels on a supervised binary classification problem. ...
1
vote
0answers
225 views

Depth of convolution layers in DNN

I was reading this morning the paper on the architecture of Alexnet and there is one thing I cannot understand. On the convolution first layer, after the ReLu activation and the response normalisation ...
1
vote
1answer
124 views

CNN Architecture

At first, this question is less about programming itself but about some logic behind the CNN architecture. I do understand how every layer works but my only question is: Does is make sense to separate ...
3
votes
1answer
2k views

Number of trainable parameters in Convolution models (Keras)

I am using keras to implement a cnn model. ...
0
votes
1answer
380 views

Classification using Convolutional neural networks for very high dimensional data

I have time series signals that are VERY high dimensional (14000 data points), the number of samples is around 1000 samples. I want to use the Convolution neural networks to classify the time series ...
1
vote
1answer
499 views

LeNet-5 Layer 5 (C5) Math

I'm having trouble understanding how the math works in the C5 convolution layer of the LeNet-5 network. Layer 4 (S4) has 16 feature maps of size 5x5. Using a convolution kernel of size 5x5 with valid ...
1
vote
1answer
78 views

Output of sentence convolution neural network yielding dimension equals to word embedding

From this article titled A Convolutional Neural Network for Modelling Sentences, it is mentioned in section 2.2 that generation the output of a convolution layer is determined by the formula of: ...
2
votes
1answer
2k views

Convolutional neural networks - What is done first? Padding or convolving?

Convolutional neural networks - What is done first? Padding or convolving? Suppose the code below : ...
-1
votes
1answer
78 views

Why is convolution operation an integral?

I was reading the wonderful book on Deep Learning (http://www.deeplearningbook.org) trying to understand convolution. It uses an example in the convolution chapter (http://www.deeplearningbook.org/...
3
votes
0answers
197 views

What do the forward and backward propagation algorithms look like for convolutional neural networks

I've tried writing convolutional neural networks a few times and I always, always, always fail. It would be really useful if someone could write out the forward and back propagation algorithms in ...
3
votes
2answers
2k views

How to handle even and odd convolutional filter sizes and images

Is there a rule of thumb for determining the size of a convolutional filter given the shape of the input? Specifically, if you want to do a 1D convolution over an even-length vector, does the kernel ...
0
votes
2answers
1k views

Convolution operator in CNN and how it differs from feed forward NN operation?

I understand that the architecture of Convolutional Neural Networks (CNN) and Feed forward (FNN) are quite different. And that CNNs use pooling and filters of shared weights over a patch of the image. ...
4
votes
2answers
6k views

2D convolution with depth

Lets say I have a convolutional neural network where my input images are of dimensions 25x25x3 (3 depth channels for colour) and pass it through a convolution layer of 5 kernels, each 3x3 The depth ...
14
votes
2answers
3k views

Are there mathematical reasons for convolution in neural networks beyond expediency?

In convolutional neural networks (CNN) the matrix of weights at each step gets its rows and columns flipped to obtain the kernel matrix, before proceeding with the convolution. This is explained on a ...
3
votes
1answer
899 views

What are the units of a convolution?

The convolution of $f$ and $g$ is defined as $ (f * g )(t) \, \stackrel{\mathrm{def}}{=}\ \int_{-\infty}^\infty f(\tau)\, g(t - \tau) \, d\tau $. Let's say that $f(t)$ and $g(t)$ have units of, say, ...
2
votes
0answers
78 views

Convolve Gamma distribution with Triangle distribution?

I am working on the use of distributed delay applied to pharmacometric models. Specifically, the delay kernel I am interested in is the Gamma distribution, with non-integer shape. The historical ...
1
vote
1answer
2k views

CNN kernels updates/initialization?

First, let's assume we are in the context of a image classification using a CNN. I understand that different kernels are applied to the image (or in succession, depending on the depth of the network)....
2
votes
0answers
53 views

Proof Estimator is Locally Regular

In order to rely on Hajek-Le Cam Convolution Theorem, I tried to show that a given root-n consistent estimator was locally regular. In particular, suppose $\{T_n\}$ is a $\sqrt{n}$-consistent ...
2
votes
1answer
2k views

What's the receptive field of a stack of dilated convolutions?

I'm reading the Wavenet paper which says: Stacked dilated convolutions enable networks to have very large receptive fields with just a few layers, while preserving the input resolution ...
1
vote
0answers
497 views

Weighting a probability density function (PDF) using another PDF

Say I have one PDF given by $$ f(r) = \begin{cases} \frac{3 r^{2}}{8}, \text{if } 0 \leq r \leq 2\\ 0\text{, otherwise} \end{cases} $$ describing the distribution of a continuous random variable $R$...
3
votes
2answers
1k views

How to update kernel values in a convolutional layer during backward pass?

I started coding backpropagation for a simple convnet and had some troubles understanding the algorithm. I do get the idea of weight update based on gradients, but because the filter kernel parameters ...
2
votes
0answers
531 views

How to get continuous output with Convolutional network in Keras?

I'm a new user in using convolutional neural networks with keras. I have a code to classify set of images into 2 classes [0,1] using CNN in keras but I need to convert this code to get continuous ...
14
votes
2answers
18k views

How to convert fully connected layer into convolutional layer? [duplicate]

When using a fully-connected network (FCN), I have problem understanding how fully-connected (FC) layer to convolutional layer conversion actually works, even after reading http://cs231n.github.io/...
1
vote
0answers
618 views

How is AlexNet MaxPooling implemented before Conv2 layer?

Can someone please explain how the maxpooling layer is implemented before the conv2 layer in AlexNet? The output of the first conv layer (which is 96 11x11x3 convolution) is 55x55x48 (times 2). Now ...
3
votes
0answers
129 views

polar analog of cartesian cross-correlation function

Background: The cross-correlation function, wrapped in frequency domain convolution, is used in particle image velocimetry to allow sub-pixel metrology. It is also used in convolutional neural ...
8
votes
0answers
9k views

How does Max Pooling handle Odd Image Dimensions? [closed]

For the even image dimension case, max pooling is simple to understand - it simply performs convolution over the image with the max operator with a $x$-by-$x$ kernel with a stride of $x$. However for ...
5
votes
1answer
261 views

Is there a nice way to visualize the convolution of two random variables?

It is easy for me to visualize the distribution of a random variable by drawing its density function. Suppose I have two independent random variables now. I can plot the densities and visualize how ...
2
votes
0answers
281 views

Problems understanding “equivariance to translation” example in deep learning book by Goodfellow et al

I am trying to understand the following part about equivariance to translation from the deep learning book by Goodfellow, Bengio and Courville (chapter 9.2, page 338-339): To say a function is ...
2
votes
0answers
128 views

Would you interpret this image as a correct deconvolution process?

I'm applying the function conv2d_grad_wrt_inputs in theano to deconv a feature map into the original image. In the figure below the first image to the left is the ...
2
votes
0answers
56 views

Would whitening correlated inputs improve kernel estimates in a Volterra expansion?

I’m modelling a discrete-time 1-dimensional signal $y(t)$ as a sum of two input variables $x_1(t), x_2(t)$ convolved with their respective kernels plus noise $y=x_1*g_1+x_2*g_2+\eta$ and I’m ...
2
votes
1answer
361 views

How does the loss backpropagate through the convolutional layer in CNN during backpropagation?

This maybe a tough and confusing (the subject itself is confusing) question. I understand how forward pass works in a typical multi-layer CNN (with multiple convolution, pooling, and ReLU). How does ...
6
votes
1answer
1k views

How do gamma distributions add and what would that model?

Density distributions add by convolution, and the result is also a density distribution. So writing this in the time domain, w.l.o.g., the question becomes how do we take a faster gamma distribution: ...
1
vote
1answer
624 views
2
votes
1answer
171 views

Convolution equality

If $k$ is an squared or absolutely integrable kernel are the belo equalities true ? $$z(s)=\int_{R}^{} \! k(u-d) x(u).du \ \ =\int_{R}^{} \! k(u+d) x(u).du \ \ $$ and $$\int_{R}^{} \! k(u-d) k(u-d^{...