Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.

Filter by
Sorted by
Tagged with
34
votes
10answers
35k views

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution, whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$; the arithmetic sum ...
25
votes
1answer
4k views

“Kernel density estimation” is a convolution of what?

I am trying to get a better understanding of kernel density estimation. Using the definition from Wikipedia: https://en.wikipedia.org/wiki/Kernel_density_estimation#Definition $ \hat{f_h}(x) = \...
114
votes
6answers
78k views

What does 1x1 convolution mean in a neural network?

I am currently doing the Udacity Deep Learning Tutorial. In Lesson 3, they talk about a 1x1 convolution. This 1x1 convolution is used in Google Inception Module. I'm having trouble understanding what ...
10
votes
2answers
2k views

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to ...
5
votes
2answers
1k views

general solution sum of two uniform random variables aY+bX=Z?

is there a general solution to that? I have seen simple examples for Y+X=Z but I was wondering how this would be with rescaling?
6
votes
3answers
2k views

Sum of normal independent random variables with coefficients

I'm trying to wrap my head around linear transformations to random variables (with coefficients > 1). Consider the two random and independent variables $X$ and $Y$ where: $$X \sim \mathcal{N}(0,1)\...
3
votes
1answer
1k views

CDF of Z=XY with X~Uniform(0.5,1.5) and Y~Uniform(0.8,1.5)

I am looking for the CDF of the product of two independent random variables (X and Y) with uniform distributions. Both random variables uniform distributions have interval boundaries (upper and lower ...
39
votes
4answers
35k views

What is translation invariance in computer vision and convolutional neural network?

I don't have computer vision background, yet when I read some image processing and convolutional neural networks related articles and papers, I constantly face the term, ...
11
votes
4answers
16k views

The sum of independent lognormal random variables appears lognormal?

I'm trying to understand why the sum of two (or more) lognormal random variables approaches a lognormal distribution as you increase the number of observations. I've looked online and not found any ...
6
votes
1answer
1k views

How do gamma distributions add and what would that model?

Density distributions add by convolution, and the result is also a density distribution. So writing this in the time domain, w.l.o.g., the question becomes how do we take a faster gamma distribution: ...
6
votes
1answer
3k views

Sum of truncated normal with two normal distributions

Suppose I have one normal distribution $W \sim N(\mu_{w},\sigma_{w}^2)$ with a known cuttoff point (percentile) on this distribution called $c$. The first part of $W \in [-\infty,c[$ needs to be ...
4
votes
1answer
385 views

Maximum of a probability vector distributed as a Dirichlet variate

Let $p_1, p_2, \ldots \sim \text{Dirichlet}(\alpha_1, \alpha_2, \ldots)$. What is the distribution of $\max(p_1, p_2, \ldots)$? I have searched for the order statistics of the Dirichlet distribution ...
6
votes
1answer
425 views

Distribution of Quotient of 2 dependent random variables

Well , I have the following problem.. Let $X_1,\cdots ,X_{2n}$ be iid $N(0,1)$ random variables. Define $$U_n=\left({X_1\over X_2}+{X_3\over X_4}+\cdots +{X_{2n-1}\over X_{2n}}\right)$$ $$V_n=X_1^...
4
votes
1answer
496 views

What is the convolution of a normal distribution with a gamma distribution?

Is there a closed form expression for the convolution of a normal distribution (ND) with a gamma distribution (GD)? There does not seem to be a direct method of solving this convolution.
3
votes
1answer
434 views

PDF of a sum using convolution

Let $X$ and $Y$ be independent, continuous random variables. The PDF of a sum $W=X+Y$ is given by the convolution formula $$ f_{X+Y} (w) = \int_{-\infty}^{\infty} f_X(x) f_Y(w-x)\,\,dx$$ In this ...
2
votes
1answer
40 views

How many input pixels influence output pixel in an FCN type architecture?

Let's say I have 8 back to back convolutional layers with zero padding such that the input and output dimensions are the same. There are no max-pooling layers between the layers. All the layers use a ...
16
votes
3answers
11k views

What does the convolution step in a Convolutional Neural Network do?

I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here ...
16
votes
2answers
855 views

A dynamical systems view of the Central Limit Theorem?

(Originally posted on MSE.) I have seen many heuristic discussions of the classical central limit theorem speak of the normal distribution (or any of the stable distributions) as an "attractor" in ...
4
votes
2answers
7k views

Oscillating validation accuracy for a convolutional neural network?

My CNN training gives me weird validation accuracy result. When it comes to 2.5,3.5,4.5 epochs, the validation accuracy is higher (meaning only need to go over half of the batches and I can reach ...
30
votes
2answers
1k views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron (pixel/parameter) ...
10
votes
2answers
15k views

Sum of Bernoulli variables with different success probabilities [duplicate]

Let $x_i$ be independent Bernoulli random variables with success probabilities $p_i$. That is, $x_i=1$ with probability $p_i$ and $x_i=0$ with probability $1-p_i$. Is there a closed expression or an ...
9
votes
1answer
3k views

Distribution of sum of exponentials

Let $X_1$ and $X_2$ be independent and identically distributed exponential random variables with rate $\lambda$. Let $S_2 = X_1 + X_2$. Q: Show that $S_2$ has PDF $f_{S_2}(x) = \lambda^2 x \text{e}^{-...
3
votes
1answer
2k views

How to compute the sum of a mixture distribution with another distribution?

I need to find the pdf of x, $f_x(x)$ which is the sum of two random variables $u$ and $w$ and they are independent. I have found the pdf but I am unsure if it is correct or not, the expression is ...
5
votes
1answer
692 views

Intuition for why sum of gaussian RVs is different from gaussian mixture

I know that in the case of Gaussian mixture, the "intuition" is that you're drawing from a PDF which itself is just a sum of weighted Gaussian PDFs. I don't understand the intuition behind how the ...
3
votes
2answers
2k views

Proof of Algebraic Formula for the Sum of Two-Dice Toss as a Convolution

To figure out exactly the expected frequency of a given sum in a dice toss (given a certain number of dice and sides/dice), the following formula is posted here by @Glen_b (adapted to dice of six ...
2
votes
0answers
156 views

About kernel based estimates

Kernel based operations are common in a variety of applications, such as image processing (e.g., blurring), generating smoothed estimation maps, and so on. A common approach is to select four ...
2
votes
1answer
73 views

Sum of N two-component mixture variates

I have a list of random variables, $X_1$, $X_2$, ..., $X_N$, associated with binary random variables $A_i$ such that $P(A_i) = \pi$ is known. I also know that, for all $i$ $$X_i|A_i\sim f(x)\\ X_i|\...
0
votes
2answers
1k views

Convolution operator in CNN and how it differs from feed forward NN operation?

I understand that the architecture of Convolutional Neural Networks (CNN) and Feed forward (FNN) are quite different. And that CNNs use pooling and filters of shared weights over a patch of the image. ...
3
votes
2answers
123 views

Sum of Independent Binomials

Let $X \sim \text{Bin}(m,p)$ and $Y \sim \text{Bin}(n,p)$. I want to find the distribution of $W = X + Y$ using a convolution theorem that says: $$f_W(w) = \sum\limits_x f_X (x) f_Y(w-x).$$ So using ...
2
votes
1answer
905 views

Padding and stride in backpropagation of a conv net

I am trying to implement the back-propagation of a simple convolutional network. Specifically I understand that one of the steps is the convolution of the gradients coming from the next layer, with ...
2
votes
0answers
233 views

How to explain certain patterns appearing after kernel averaging?

Having a 2D map filled uniformly by random values (Figure:top-left), the next maps are kernel averaged with a kernel of sizes ...
0
votes
2answers
2k views

How to work multiple filter region sizes: 2, 3 and 4 in CNN?

I mention learn convolutional neural networks (CNN) for classification of sentences made by Yoonkim. I am still confused about the size of the filter and how convolution works . What do ...
0
votes
1answer
266 views

Does the sum of two independent exponentially distributed random variables with different rate parameters follow a gamma distribution?

short question. Suppose we have two independent exponentially distributed random variables with means $400$ and $200$, so that their respective rate parameters are $1/400$ and $1/200$. Do these ...