Mathematical representation of 1D convolution

Question

How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding?

For instance, I can write

import torch

input1d = torch.tensor([[[1,2,3,4,5]]])
filter1d = torch.tensor([[[4,3]]])

torch.nn.functional.conv1d(input1d, filter1d)

Output:

tensor([[[10, 17, 24, 31]]])

This is equivalent to

torch.einsum("abcd,abd->abc", input1d.unfold(2,2,1), filter1d)

How does this translate to mathematical operations? I am particularly uncertain about input1d.unfold(2,2,1).

I have the following, but I am quite sure it is incorrect $$ x = \mathrm{input1d} \in \mathbb{R}^{1 \times 1 \times 5} \\ w = \mathrm{filter1d} \in \mathbb{R}^{1 \times 1 \times 2} \\ \int_{-\infty}^{\infty} \sum_{d} x_{abcd}\,w_{abd} $$

Answer 1

So a few things here:

Firstly, it is worth mentioning for the sake of transparency that torch.nn.functional.conv1d is more strictly cross-correlation rather than convolution, which involves flipping the filter, in a more broad usage. However, for CNN applications, the distinction is not important, and so the term convolution is overwhelmingly overloaded to mean cross-correlation (no flip).

The second thing that is worth mentioning is that ``mathematical operations'' can mean a few things as well. A relatively straightforward approach would be using

$$(\vec{a}\star\vec{b})[i] = \vec{a}\left[i:i+\text{len}(\vec{b})\right]\cdot \vec{b}$$

where $\vec{a}\star\vec{b}$ is the desired result of cross-correlating $\vec{a}$ and $\vec{b}$ (given as a vector-describing function $\vec{a}\star\vec{b}:\mathbb{Z} \rightarrow\mathbb{R}$ that maps the vector indices in $\mathbb{Z}$ to results in $\mathbb{R}$). $:$ and $\cdot$ are range-indexing and dot product operations, which can be broken down into summation notation as desired.

Incorporating stride, this is

$$\text{corr}(\vec{a}, \vec{b}, \text{stride})[i] = \vec{a}\left[i\cdot\text{stride}:i\cdot\text{stride}+\text{len}(\vec{b})\right]\cdot \vec{b}$$

After breaking this down into summation notation, you should be able to extend the dot product operation to consider padded zeros (or more precisely, only sum/multiply existing elements). I will circle back to address this and your question involving matrix multiplication and reindexing when I'm free from work.