Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.

Filter by
Sorted by
Tagged with
114
votes
6answers
78k views

What does 1x1 convolution mean in a neural network?

I am currently doing the Udacity Deep Learning Tutorial. In Lesson 3, they talk about a 1x1 convolution. This 1x1 convolution is used in Google Inception Module. I'm having trouble understanding what ...
42
votes
6answers
41k views

Importance of local response normalization in CNN

I've found that Imagenet and other large CNN makes use of local response normalization layers. However, I cannot find that much information about them. How important are they and when should they be ...
39
votes
4answers
35k views

What is translation invariance in computer vision and convolutional neural network?

I don't have computer vision background, yet when I read some image processing and convolutional neural networks related articles and papers, I constantly face the term, ...
34
votes
10answers
35k views

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution, whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$; the arithmetic sum ...
30
votes
2answers
1k views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron (pixel/parameter) ...
25
votes
1answer
4k views

“Kernel density estimation” is a convolution of what?

I am trying to get a better understanding of kernel density estimation. Using the definition from Wikipedia: https://en.wikipedia.org/wiki/Kernel_density_estimation#Definition $ \hat{f_h}(x) = \...
19
votes
6answers
20k views

Convolutional Layers: To pad or not to pad?

AlexNet architecture uses zero-paddings as shown in the pic: However, there is no explanation in the paper why this padding is introduced. Standford CS 231n course teaches we use padding to ...
16
votes
2answers
855 views

A dynamical systems view of the Central Limit Theorem?

(Originally posted on MSE.) I have seen many heuristic discussions of the classical central limit theorem speak of the normal distribution (or any of the stable distributions) as an "attractor" in ...
16
votes
3answers
11k views

What does the convolution step in a Convolutional Neural Network do?

I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here ...
14
votes
2answers
3k views

Are there mathematical reasons for convolution in neural networks beyond expediency?

In convolutional neural networks (CNN) the matrix of weights at each step gets its rows and columns flipped to obtain the kernel matrix, before proceeding with the convolution. This is explained on a ...
14
votes
2answers
19k views

How to convert fully connected layer into convolutional layer? [duplicate]

When using a fully-connected network (FCN), I have problem understanding how fully-connected (FC) layer to convolutional layer conversion actually works, even after reading http://cs231n.github.io/...
14
votes
2answers
2k views

Distribution of the convolution of squared normal and chi-squared variables?

the following problem came up recently while analyzing data. If the random variable X follows a normal distribution and Y follows a $\chi^2_n$ distribution (with n dof), how is $Z = X^2 + Y^2$ ...
13
votes
3answers
9k views

How exactly do convolutional neural networks use convolution in place of matrix multiplication?

I was reading Yoshua Bengio's Book on deep learning and it says on page 224: Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at ...
11
votes
4answers
16k views

The sum of independent lognormal random variables appears lognormal?

I'm trying to understand why the sum of two (or more) lognormal random variables approaches a lognormal distribution as you increase the number of observations. I've looked online and not found any ...
10
votes
3answers
9k views

How to classify a unbalanced dataset by Convolutional Neural Networks (CNN)?

I have a unbalanced dataset in a binary classification task, where the positives amount vs negatives amount is 0.3% vs 99.7%. The gap between positives and negatives are huge. When I train a CNN with ...
10
votes
2answers
2k views

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to ...
10
votes
2answers
441 views

Stable distributions that can be multiplied?

Stable distributions are invariant under convolutions. What sub-families $F$ of the stable distributions are also closed under multiplication? In the sense that if $f\in F$ and $g\in F $, then the ...
10
votes
2answers
15k views

Sum of Bernoulli variables with different success probabilities [duplicate]

Let $x_i$ be independent Bernoulli random variables with success probabilities $p_i$. That is, $x_i=1$ with probability $p_i$ and $x_i=0$ with probability $1-p_i$. Is there a closed expression or an ...
9
votes
1answer
7k views

Efficient convolution (in R)

I want to calculate/evaluate the convolution $$g(x)=\int_D f(x-t) \phi(t) dt,$$ where $f$ is a density and $\phi$ is a smooth function with compact support $D$. The convolution is not available in ...
9
votes
1answer
3k views

Distribution of sum of exponentials

Let $X_1$ and $X_2$ be independent and identically distributed exponential random variables with rate $\lambda$. Let $S_2 = X_1 + X_2$. Q: Show that $S_2$ has PDF $f_{S_2}(x) = \lambda^2 x \text{e}^{-...
8
votes
0answers
9k views

How does Max Pooling handle Odd Image Dimensions? [closed]

For the even image dimension case, max pooling is simple to understand - it simply performs convolution over the image with the max operator with a $x$-by-$x$ kernel with a stride of $x$. However for ...
7
votes
2answers
1k views

Why does multiplication in the frequency domain equal convolution in the time domain?

This question came in the context of understanding how to get a distribution of a sum of two iid random variables. I'm working through the top answer to this question Consider the sum of $n$ uniform ...
7
votes
2answers
4k views

In convolutional neural network, what does fully-connected layer mean?

There are convolution layers, pooling layers, and possibly a classifier layer (e.g. softmax layer) in a convolutional neural network (CNN). I heard that there is also a fully-connected layer. What ...
7
votes
2answers
13k views

Difference between Conv and FC layers?

What is the difference between conv layers and FC layers? Why cannot I use conv layers instead of FC layers?
7
votes
1answer
4k views

Proving the Convolution of PDFs gives a PDF

I am struggling with a question about the convolution of PDFs, in particular, proving that given two PDFs $f$ and $g$, then their convolution $f*g$, will also be a PDF. Proving non negativity is easy ...
7
votes
2answers
11k views

Training a convolutional neural network

Based on my research on convolution neural networks, every other layer in such a network has a subsampling operation, in which the resolution of the image is reduced so as to improve generalization of ...
7
votes
1answer
2k views

Why do people use Zero-Padding in Convolutional Neural Networks?

I am wondering why people usually pad with zeros instead of e.g., using the min-value. Zero-padding, in my opinion, makes sense if you have input images with a pixel range [0, 255] or [0, 1] (after ...
7
votes
0answers
462 views

Sum of absolute values of T random variables

Where X is a r.v. following a symmetric T distribution with 0 mean and tail parameter $\alpha$. I am looking for the distribution of the n-summed variable $ \sum_{1 \leq i \leq n}|x_i|$. $Y=|X|$ ...
6
votes
3answers
2k views

Sum of normal independent random variables with coefficients

I'm trying to wrap my head around linear transformations to random variables (with coefficients > 1). Consider the two random and independent variables $X$ and $Y$ where: $$X \sim \mathcal{N}(0,1)\...
6
votes
3answers
1k views

What makes neural network a *convolutional* neural network?

What is the difference between a Convolutional Neural Network (CNN) and an ordinary Neural Network (NN)? What does convolution mean in this context?
6
votes
1answer
1k views

How do gamma distributions add and what would that model?

Density distributions add by convolution, and the result is also a density distribution. So writing this in the time domain, w.l.o.g., the question becomes how do we take a faster gamma distribution: ...
6
votes
2answers
298 views

Convolution of random variables: unimodality of the likelihood function

Let $X_1, X_2,...X_k$ be random independent variables, each $X_i$ drawn from a Geometric distribution $\mathcal{G}(p_i)$, and let its convolution, or sum, be $Y = \sum_{i=1}^k X_i$. The likelihood ...
6
votes
1answer
3k views

Sum of truncated normal with two normal distributions

Suppose I have one normal distribution $W \sim N(\mu_{w},\sigma_{w}^2)$ with a known cuttoff point (percentile) on this distribution called $c$. The first part of $W \in [-\infty,c[$ needs to be ...
6
votes
1answer
673 views

How does backpropagation learn convolution filters?

I've understood how the backpropagation algorithm uses the partial derivatives of the weights to train a normal neural network. However, I cannot quite understand how the algorithm changes the filters....
6
votes
1answer
97 views

Express product as convolution? Or, given $f(s)$, find $g$ satisfying $f(s)=\mathbb{E}[g(X)]$ where $X\sim \mathcal{N}(0,s^2)$

Given a function $f(\mu)$ (satisfying certain properties), it is possible to find a function $g(x)$ such that $f(\mu)=\int_{-\infty}^{\infty} g(x)\phi(x-\mu) dx $, where $\phi$ is the standard normal ...
6
votes
1answer
425 views

Distribution of Quotient of 2 dependent random variables

Well , I have the following problem.. Let $X_1,\cdots ,X_{2n}$ be iid $N(0,1)$ random variables. Define $$U_n=\left({X_1\over X_2}+{X_3\over X_4}+\cdots +{X_{2n-1}\over X_{2n}}\right)$$ $$V_n=X_1^...
6
votes
1answer
409 views

Is output of Deamer deconvolution not a density?

I have a Model Y= X+e and need the density of X. The deamer package deconvolves the density for X, but if I use the simpsons rule to integrate this density, I get values which are above 1. The ...
6
votes
2answers
2k views

Convolutional neural networks backpropagation

My question is regarding the answer to this question: Training a convolution neural network It seems like the answer is saying to change all the weights in a given filter by the same amount in the ...
6
votes
1answer
277 views

Sum of truncated Gammas

I have a set of i.i.d. variables $X_i$ that are distributed according to a truncated $\text{Gamma}(\alpha,\beta)$ distribution, with support on $[0,w)$ where $w$ is a known constant. What's the ...
6
votes
2answers
909 views

Does the property of equivariance to translation of convolution layers help to learn translation-invariant features?

In some texts, people mention that the reason why convolutional neural networks are able to learn translation-invariant features are related to the property that convolution layers are equivariant to ...
6
votes
0answers
599 views

Two-sample bootstrap?

I have two independent samples of observations. From each sample I produce a statistic. Let's denote these as $\theta_1$ and $\theta_2$. I'd like to test the hypothesis that $H_0: \Theta_1=\Theta_2$, ...
5
votes
2answers
1k views

general solution sum of two uniform random variables aY+bX=Z?

is there a general solution to that? I have seen simple examples for Y+X=Z but I was wondering how this would be with rescaling?
5
votes
2answers
12k views

In convolutional neural networks, how to prevent the overfitting?

Given certain amount of labeled data, we define the net structure, such as number of layers, types of layers, the number of convolutional layers, the number of pooling layers, etc. And train the ...
5
votes
4answers
3k views

How does local connection implied in the CNN algorithm

I am trying to understand the process of Convolutional Neural Networks. Basically, I am trying to understand how does the local connection works. The first step of CNN is a convolution layer where ...
5
votes
1answer
691 views

Intuition for why sum of gaussian RVs is different from gaussian mixture

I know that in the case of Gaussian mixture, the "intuition" is that you're drawing from a PDF which itself is just a sum of weighted Gaussian PDFs. I don't understand the intuition behind how the ...
5
votes
1answer
3k views

CNN vs RNN for time series classification

I am new to neural networks and after some research i read about CNN and RNN neural networks. The data that i am having is multiple different time series of numbers. So for example instead of input 1 ...
5
votes
1answer
269 views

Is there a nice way to visualize the convolution of two random variables?

It is easy for me to visualize the distribution of a random variable by drawing its density function. Suppose I have two independent random variables now. I can plot the densities and visualize how ...
5
votes
1answer
364 views

Placement of earlier features in more complex features in CNN

I'm trying to understand convolutional neural networks better. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
5
votes
0answers
111 views

Finite sum of beta prime iid random variables

The beta prime distribution is infinitely divisible, as proved in Steutel and van Harn, 2003 (Appendix B). Sadly, in this book, there is no espression of the parameters of the distribution of n ...
4
votes
2answers
6k views

2D convolution with depth

Lets say I have a convolutional neural network where my input images are of dimensions 25x25x3 (3 depth channels for colour) and pass it through a convolution layer of 5 kernels, each 3x3 The depth ...

1 2 3 4 5 7