Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.

Filter by
Sorted by
Tagged with
1 vote
1 answer
49 views

How to calculate the Transposed Convolution?

Studying for my finals in Deep learning. I'm trying to solve the following question: Calculate the Transposed Convolution of input $A$ with kernel $K$: $$ A=\begin{pmatrix}1 & 0 & 1\\ 0 & ...
  • 171
2 votes
1 answer
13 views

Sparsity of connections in a CNN - why is it beneficial?

In his CNN video, Andrew Ng explains the benefits of convolutions over FC, and he mentions sparsity of connections as being one of them. Can someone please explain what is the benefit of sparsity when ...
1 vote
0 answers
28 views

How to explain the high accuracy and F1 score on the test set with a huge binary crossentropy loss?

I'll provide a little of introduction based on my example. I have a small collection of RGB (but 'gray-looking') brain MRI photos, divided into 2 classes: healthy and tumor. My data split looks like ...
0 votes
0 answers
11 views

does kernel or filter update through backpropagation , is require fully connected layers?

does CNN itself without FC has the ability to make backpropagation to update its filters, after comparing them with the output and calculating the loss, and then pass them through the FCC?
1 vote
1 answer
30 views

GAN artifacts on borders

not quite a math-question, but I have a doubt. I'm trying to build from scratch the Pix2pix network, on the facades dataset, and I think I finally got a good model (from the paper I borrowed just the ...
1 vote
0 answers
17 views

Why not using Convolutional pooling instead of MaxPooling to avoid invariance

I was reading the paper of [Geoffrey Hinton: Capsule network], and I watch it's talk on Youtube about the problem of Conv Network is actually the (max) pooling layer, since we don't want to be ...
1 vote
1 answer
19 views

How does fix-up initialization avoid prevent the neurons from updating in the exact same direction?

We know zero initialization is bad: Pitfall: all zero initialization. Lets start with what we should not do. Note that we do not know what the final value of every weight should be in the trained ...
  • 237
0 votes
0 answers
22 views

Validation loss is stuck at certain loss [duplicate]

I am trying to implement 1D-CNN to time-series data. Here is my data. Here is my dataframe: I am trying to forecast each of these futures. I am following the TensorFlow forecasting guide. Based on ...
  • 152
0 votes
0 answers
30 views

Sum of iid Exponential observations then subtracting the minimum of the observations [duplicate]

Consider $n$ iid $X_1,...,X_n \sim Exp(1)$. My goal is to find the density of $\sum (X_i - X_{(1)})$. My attempt If we write out the entire summation in order statistics, we get $X_{(1)}-X_{(1)} + X_{(...
  • 89
0 votes
0 answers
81 views

Deformable Convolution implementation in tensorflow2

Is there any good implementation of Deformable Convolution in tensorflow2? Deformable Convolutional Networks Deformable ConvNets v2: More Deformable, Better Results
0 votes
1 answer
52 views

Find pdf of X+Y [duplicate]

Let X ∼ Exp(λ) and Y ∼ Exp(μ) be two independent exponential random variables, where λ, μ > 0. Find the probability density function of X + Y if λ ̸= μ. I have successfully find ans if λ = μ, but ...
  • 1
4 votes
2 answers
158 views

When does the sum of two $t$-distributed random variables follow a $t$ distribution?

In the scope of a project, I need to find the sum of two independent $t$-distributions. I know that in the general case, the sum of two $t$-distributed random variables is not $t$-distributed. However,...
  • 337
1 vote
0 answers
23 views

How general is this property about correlation and the sum of two normal RVs?

(Cross-posted from math stack exchange as I didn't get any responses there) Given a random vector $(X_1,X_2)$ that is jointly normal with means / sd's $\mu_1,\mu_2, \sigma_1,\sigma_2$ and correlation ...
  • 11
1 vote
0 answers
24 views

Covariance of a convolution between a gaussian random walk and white noise [closed]

I want to compute the covariance of $$U_t:=\sum_{l=-L}^{l=L} (X_l-X_{l-1})X_{l-t}$$ with $X_t$ defined as : \begin{align*} X_0&=0 \\ X_t&=X_{t-1}+\epsilon_t \end{align*} $t=1,2,...$...
  • 11
1 vote
1 answer
96 views

Issue about both results in agreement with 2 different ways to compute variance of a random variable : weighted chisquared vs Gamma distributions

1.) I am interested in computing the variance of this observable $O$ involving the coefficients of spherical harmonics $a_{\ell m}$ and the $C_{\ell}$ which is the variance of an $a_{\ell m}$ : $$O=\...
1 vote
1 answer
30 views

3D convolutions and jitter

I want to process a sequence of cropped and aligned face images from a video with a neural network. I am considering a use of 3D convolutions in order to capture the spatiotemporal dependency. However,...
0 votes
0 answers
20 views

Why does MobileNet Architecture start with a Standard Convolution?

I am trying to understand the design choices behind the MobileNet architecture. (pdf available on the right). The authors use Depthwise Separable Convolutions as a replacement for Classical ...
  • 1
2 votes
1 answer
146 views

How to count the parameters in a convolution layer?

I'm preparing for an exam in Computer Vision. I came across with the following question from one of the exams: What is the number of parameters of a convolution layer in a neural network, when the ...
  • 171
0 votes
1 answer
92 views

How to obtain the density of a sum of independent discrete random variables?

Let's $X_1, X_2, ..., X_n$, $n=1,2,...$ are independent discrete random variables. It is necessary to find the distribution law of the their sum: $p(k) =P(X_1 + X_2 + ... + X_n = k), k=0, 1, 2, ... $ ...
  • 850
0 votes
0 answers
22 views

Recovering the distribution of an independent variable of a probit model

I have the following model: $$ y_i^* = \beta_0 + \beta_1 x_i + \epsilon_i,$$ where I assume that $\epsilon \sim N(0,1)$ and $x \sim N(\mu, \sigma^2)$. I dont observe $y^*$, only $y_i=1 \text{ if } y_i^...
  • 29
0 votes
0 answers
16 views

CNN model accuracy is increasing really slowly [duplicate]

I am training a CNN model from scratch on the Caltech101 dataset. The accuracy of the model is increasing very slowly after the 5th epoch. Shown below is the accuracy and loss curves of the model for ...
1 vote
1 answer
89 views

Why my cdf of the convolution of n exponential distribution is not in the range(0,1)?

I assume that there are n exponential distribution that $x_i$ ~ $Exp(\lambda_i)$, i=1..n, and I want to calculate the cumulative distribution of $S=x_1+x_2+...+x_n$, the convolution of n exponential ...
1 vote
1 answer
71 views

Training accuracy growing way faster than validation accuracy?

i've been trying to solve this classification problem using a convolutional neural network for some days but no matter what I do I can't seem to find the correct hyperparameters and configuration to ...
0 votes
0 answers
16 views

What prevents my PyTorch convolutional auto-encoder to converge on some initializations? [duplicate]

I built a small auto-encoder for greyscale images. It is there to make some tests, so I train it often, and I have a strange behavior. On some initialisations, it does not converge. I mean, the MSE ...
  • 111
0 votes
0 answers
13 views

Convolution of a conditional PDF with itself [duplicate]

Let's say that $x$ is a random variable which follows a conditional probability density function $f(x|N)$, where $N$ is an integer parameter. If the functional form of $f(x|N=1)$ is known, then one ...
  • 409
0 votes
0 answers
62 views

convolving a conditional pdf with itself

Let's say that x is a random variable which follows a conditional probability density function $f(x|N)$, where $N$ is an integer parameter. If the functional form of $f(x|N=1)$ is known, then one can ...
  • 409
0 votes
0 answers
30 views

Is there an explanation for a classifier achieving high F1 scores, but having still high CrossEntropyLoss?

I am training a CNN classifier on a balanced dataset (around 35k examples for each label) with 13 classes. The model seems to achieve high F1 scores from the first batches; The F1 score for each class ...
  • 3
2 votes
0 answers
23 views

How is it ensured that a CNN does not learn the same filter multiple times?

Let's say we have a dataset of pictures displaying straight vertical lines with different translations or nothing (just an example), and we apply a CNN to that. We choose that the CNN should learn N ...
0 votes
0 answers
22 views

How is derivative calculated for Grad-CAM if the final output is multidimensional?

For Grad-CAM, the derivative of the final output is found with respect to the elements of the channel considered Selvaraju et al. 2019. But if the output is a multidimensional matrix how is the ...
1 vote
0 answers
85 views

Kernel_size for rgb images in cnn?

I came across a cnn code of rgb images where kernel_size was mentioned only 3 not 3,3,3. So does 3 means 3,3,3. and for greyscale images kernel size was mentioned 3,3 so for grey scale images 3,3 I ...
0 votes
1 answer
71 views

Complexity of 1D CNN and 2D CNN

Are the computational complexity of 1D CNN and 2D CNN the same? If not what are their computational complexity and what is the best way to compute them? Considering both forward and backward ...
  • 101
1 vote
0 answers
34 views

Working with modalities (sensor data) with different frequencies

Accelerometer data: 119 Hz. Gyroscope data: 119 Hz. Magnetometer data: 20 Hz. I would like to build a 2D-CNN. I have sensors with different sampling rates. Rather than extracting a feature, how I can ...
  • 152
2 votes
0 answers
19 views

Can we recover joint distribution from a continuous range of convolutions?

Given a correct model of the joint distribution of a random vector $(X,Y)$, we can derive (though not necessarily in closed form) the correct distribution model for $pX+(1-p)Y$ for any $p\in[0,1]$. ...
0 votes
0 answers
204 views

How does max pooling reduce the number of parameters to be learnt in a CNN if we already have parameter sharing?

I've been a bit confused about max pooling for a little while but I've finally understood why it provides more generalisation since you're essentially getting rid of less useful information and ...
0 votes
0 answers
110 views

Convolution of Binomial and Poisson Distributions?

I am currently working through the paper Estimation of Probability of Defaults (PD) for Low-Default Portfolios: An Actuarial Approach In Section 2 of this paper, the author provides the following ...
1 vote
0 answers
61 views

Computing an Exponential Moving Average

According to the Wikipedia page on moving averages, "This is also why sometimes an EMA is referred to as an $N$-day EMA. Despite the name suggesting there are $N$ periods, the terminology only ...
1 vote
0 answers
53 views

Guessing filters from responses to step signals

Consider a signal $X$ filtered by a kernel $p$ with finite support $[t_0,t_1]$ and $\int_{t_0}^{t_1}p(t)\,\text{d}t = 1$, yielding the response function $$\overline{X}(T) = \int_{t_0}^{t_1} X(T + t)\ ...
0 votes
0 answers
101 views

Discrete convolution with circular matrices

Cross-correlation (read convolution) is the process of sliding a kernel across the input image and each step multiplying the elements and summing the results (i.e. matrix multiplication). PyTorch has ...
  • 73
0 votes
0 answers
35 views

Conditional expectation of X given X+Y [duplicate]

X and Y are two independent variables, X ~ exp(a), Y ~ exp(a). I need to find E(X|X+Y). I tried to calculate by definition, but it did not lead to success. Maybe there is another, more convenient way ...
  • 9
2 votes
1 answer
123 views

How doesn't a dilated convolution lose information?

In the example below (source), we see the difference between stride and dilation in CNNs. The explanation as quoted: "Using a dilated convolution increases the size of the receptive field ...
  • 649
2 votes
0 answers
28 views

How could I downsize CNN?

Let's take a CNN with an input of size (512, 512) and an output of the same size. Now imagine I want to feed an image of size (256, 256) to the network. Would it be possible to do so without upscaling ...
  • 21
0 votes
0 answers
179 views

Are convolutional autoencoders required to have symmetric encoders and decoders?

I am a newer to deep learning. Recently I am studying the convolutional autoencoder (CAE). I found the architectures built with keras and matlab are a little different. In particular, the architecture ...
0 votes
0 answers
8 views

Make output of a sequential model self-consistent

I'm training a sequential model on the following type of sequence (just showing the target labels here): ...
  • 531
2 votes
1 answer
298 views

Mathematical representation of 1D convolution

How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding? For instance, I can write ...
  • 73
1 vote
1 answer
76 views

Spectral Graph Convolutions: What are the spectral filters functions

I am trying to understand the mathematical meaning of one of the steps that appear in the Convolution Theorem (Step 4 here). To give some context, this is related to applying the convolution theorem ...
2 votes
0 answers
57 views

When is $\sum Z_i \sim \sqrt{n} Z_i$?

If $X_i$ are independently and identically distributed $N(0,\sigma^2)$ then $Y=\sum X_i \sim N(0,n\sigma^2)$, i.e. $\sum X_i \sim \sqrt{n}X_i$. That raises two questions: Is a zero-mean normal ...
  • 32.4k
0 votes
0 answers
14 views

if i need a kernel to detect cones shape.. how to do so?

i'm working on a project of cones detection from lidar point cloud, I have got an idea to use Hough transform and I'm using convolution for voting principle for cone detection and I already ...
0 votes
0 answers
14 views

A question about Convolutional Neural Networks, equivariance and parameter sharing

I have a couple of questions acbout Convolutional Neural Networks and I'm struggling to give an answer. Q1 Let's say I have $[3 \times 32 \times 32]$ image (with three channels) and I apply a first ...
1 vote
1 answer
198 views

What is the best way to feed IMU data to CNN?

I took the Introduction to Embedded Machine Learning course, which is provided by Shawn Hymel, on Coursera. While talking about sensor fusion, he made the following statement for the following diagram:...
  • 152
1 vote
1 answer
2k views

Why CNN is suitable for time-series data? [duplicate]

I am confused by the statements that I came across in two different papers. The statement from the paper titled as "Detecting Cyber Attacks in Industrial Control Systems Using Convolutional ...
  • 152

1
2 3 4 5
9