What is the difference between Conv1D and Conv2D?

I was going through the keras convolution docs and I have found two types of convultuion Conv1D and Conv2D. I did some web search and this is what I understands about Conv1D and Conv2D; Conv1D is used for sequences and Conv2D uses for images.

I always thought convolution nerual networks were used only for images and visualized CNN this way

A image is considered as a large matrix and then a filter will slide over this matrix and compute the dot product. This I believe what keras mentions as a Conv2D. If Conv2D works this way then what is the mechanism of Conv1D and how we can imagine its mechanism?

• Take a look at this answer. Hope this helps. Oct 28, 2017 at 0:03

5 Answers

I'd like to explain the difference visually and in detail(comments in code) and in a very easy approach.

Let's first check the Conv2D in TensorFlow.

c1 = [[0, 0, 1, 0, 2], [1, 0, 2, 0, 1], [1, 0, 2, 2, 0], [2, 0, 0, 2, 0], [2, 1, 2, 2, 0]]
c2 = [[2, 1, 2, 1, 1], [2, 1, 2, 0, 1], [0, 2, 1, 0, 1], [1, 2, 2, 2, 2], [0, 1, 2, 0, 1]]
c3 = [[2, 1, 1, 2, 0], [1, 0, 0, 1, 0], [0, 1, 0, 0, 0], [1, 0, 2, 1, 0], [2, 2, 1, 1, 1]]
data = tf.transpose(tf.constant([[c1, c2, c3]], dtype=tf.float32), (0, 2, 3, 1))
# we transfer [batch, in_channels, in_height, in_width] to [batch, in_height, in_width, in_channels]
# where batch = 1, in_channels = 3 (c1, c2, c3 or x[:, :, 0], x[:, :, 1], x[:, :, 2] in the gif), in_height and in_width are all 5(the sizes of the blue matrices without padding)
f2c1 = [[0, 1, -1], [0, -1, 0], [0, -1, 1]]
f2c2 = [[-1, 0, 0], [1, -1, 0], [1, -1, 0]]
f2c3 = [[-1, 1, -1], [0, -1, -1], [1, 0, 0]]
filters = tf.transpose(tf.constant([[f2c1, f2c2, f2c3]], dtype=tf.float32), (2, 3, 1, 0))
# transfer the [out_channels, in_channels, filter_height, filter_width] to [filter_height, filter_width, in_channels, out_channels]
# out_channels is 1(in the gif it is 2 since here we only use one filter W1), in_channels is 3 because data has three channels(c1, c2, c3), filter_height and filter_width are all 3(the sizes of the filter W1)
# f2c1, f2c2, f2c3 are the w1[:, :, 0], w1[:, :, 1] and w1[:, :, 2] in the gif
output = tf.squeeze(tf.nn.conv2d(data, filters, strides=2, padding=[[0, 0], [1, 1], [1, 1], [0, 0]]))
# this is just the o[:,:,1] in the gif
# <tf.Tensor: id=93, shape=(3, 3), dtype=float32, numpy=
# array([[-8., -8., -3.],
#        [-3.,  1.,  0.],
#        [-3., -8., -5.]], dtype=float32)>


And the Conv1D is a special case of Conv2D as stated in this paragraph from the TensorFlow doc of Conv1D.

Internally, this op reshapes the input tensors and invokes tf.nn.conv2d. For example, if data_format does not start with "NC", a tensor of shape [batch, in_width, in_channels] is reshaped to [batch, 1, in_width, in_channels], and the filter is reshaped to [1, filter_width, in_channels, out_channels]. The result is then reshaped back to [batch, out_width, out_channels] (where out_width is a function of the stride and padding as in conv2d) and returned to the caller.

Let's see how we can transfer Conv1D to a Conv2D problem. Since Conv1D is usually used in NLP scenarios, we can illustrate that in the below NLP problem.

cat = [0.7, 0.4, 0.5]
sitting = [0.2, -0.1, 0.1]
there = [-0.5, 0.4, 0.1]
dog = [0.6, 0.3, 0.5]
resting = [0.3, -0.1, 0.2]
here = [-0.5, 0.4, 0.1]
sentence = tf.constant([[cat, sitting, there, dog, resting, here]]
# sentence[:,:,0] is equivalent to x[:,:,0] or c1 in the first example and the same for sentence[:,:,1] and sentence[:,:,2]
data = tf.reshape(sentence), (1, 1, 6, 3))
# we reshape [batch, in_width, in_channels] to [batch, 1, in_width, in_channels] according to the quote above
# each dimension in the embedding is a channel(three in_channels)
f3c1 = [0.6, 0.2]
# equivalent to f2c1 in the first code snippet or w1[:,:,0] in the gif
f3c2 = [0.4, -0.1]
# equivalent to f2c2 in the first code snippet or w1[:,:,1] in the gif
f3c3 = [0.5, 0.2]
# equivalent to f2c3 in the first code snippet or w1[:,:,2] in the gif
# filters = tf.constant([[f3c1, f3c2, f3c3]])
# [out_channels, in_channels, filter_width]: [1, 3, 2]
# here we also have only one filter and also three channels in it. Please compare these three with the three channels in W1 for the Conv2D in the gif
filter1D = tf.transpose(tf.constant([[f3c1, f3c2, f3c3]]), (2, 1, 0))
# shape: [2, 3, 1] for the conv1d example
filters = tf.reshape(filter1D, (1, 2, 3, 1))  # this should be expand_dim actually
# transpose [out_channels, in_channels, filter_width] to [filter_width, in_channels, out_channels]] and then reshape the result to [1, filter_width, in_channels, out_channels] as we described in the text snippet from Tensorflow doc of conv1doutput
output = tf.squeeze(tf.nn.conv2d(data, filters, strides=(1, 1, 2, 1), padding="VALID"))
# the numbers for strides are for [batch, 1, in_width, in_channels] of the data input
# <tf.Tensor: id=119, shape=(3,), dtype=float32, numpy=array([0.9       , 0.09999999, 0.12      ], dtype=float32)>


Let's do that using Conv1D(also in TensorFlow):

output = tf.squeeze(tf.nn.conv1d(sentence, filter1D, stride=2, padding="VALID"))
# <tf.Tensor: id=135, shape=(3,), dtype=float32, numpy=array([0.9       , 0.09999999, 0.12      ], dtype=float32)>
# here stride defaults to be for the in_width


We can see that the 2D in Conv2D means each channel in the input and filter is 2 dimensional(as we see in the gif example) and 1D in Conv1D means each channel in the input and filter is 1 dimensional(as we see in the cat and dog NLP example).

Convolution is a mathematical operation where you "summarize" a tensor or a matrix or a vector into a smaller one. If your input matrix is one dimensional then you summarize along that on dimensions, and if a tensor has n dimensions then you could summarize along all n dimensions. Conv1D and Conv2D summarize (convolve) along one or two dimensions.

For instance, you could convolve a vector into a shorter vector as followss. Get a "long" vector A with n elements and convolve it using the a weight vector W with m elements into a "short" (summary) vector B with n-m+1 elements: $$b_i=\sum_{j=m-1}^0 a_{i+j}*w_j$$ where $$i=[1,n-m+1]$$

So, if you have vector of length n, and your weight matrix is also length n $$w_i=1/n$$, then the convolution will produce a scalar or a vector of length 1 equal to the average value of all values in the input matrix. It's a sort of degenerate convolution if you wish. If the same weight matrix is one shorter than the input matrix, then you get a moving average in the output of length 2 etc.

$$\begin{bmatrix} a:&a_1 & a_2 & a_3\\ w:&1/2 & 1/2&\\ w:&&1/2 & 1/2\\ \end{bmatrix}=\begin{bmatrix} b:&\frac{a_1+a_2} 2 & \frac{a_2+a_3} 2 \end{bmatrix}$$

You could do the same to 3 dimensional tensor (matrix) the same way: $$b_{ikl}=\sum_{j_1=m_1-1\\j_2=m_2-1\\j_3=m_4-1}^{0} a_{i+j_1,k+j_2,l+j_3}*w_{j_1j_2j_3}$$ where $$i=[1,n_1-m_1+1],k=[1,n_2-m_2+1],l=[1,n_3-m_3+1]$$

I will be using a Pytorch perspective, however, the logic remains the same.

When using Conv1d(), we have to keep in mind that we are most likely going to work with 2-dimensional inputs such as one-hot-encode DNA sequences or black and white pictures.

The only difference between the more conventional Conv2d() and Conv1d() is that latter uses a 1-dimensional kernel as shown in the picture below.

In here, the height of your input data becomes the “depth” (or in_channels), and our rows become the kernel size. For example,

import torch
import torch.nn as nn

tensor = torch.randn(1,100,4)
output = nn.Conv1d(in_channels =100,out_channels=1,kernel_size=1,stride=1)(tensor)
#output.shape == [1,1,4]


We can see that the kernel automatically spans to the height of the picture (just as in Conv2d() the depth of the kernel automatically spans the image’s channels) and therefore all we are left to give is the kernel size with respect to the span of the rows.

We just have to remember that if we are assuming a 2-dimensional input, our filters become our columns and our rows become the kernel size.

In summary, In 1D CNN, kernel moves in 1 direction. Input and output data of 1D CNN is 2 dimensional. Mostly used on Time-Series data.

In 2D CNN, kernel moves in 2 directions. Input and output data of 2D CNN is 3 dimensional. Mostly used on Image data.

In 3D CNN, kernel moves in 3 directions. Input and output data of 3D CNN is 4 dimensional. Mostly used on 3D Image data (MRI, CT Scans).

You can find more details here: https://medium.com/@xzz201920/conv1d-conv2d-and-conv3d-8a59182c4d6

This 1d convolution is cost saver, it work in the same way but assume a 1 dimension array that makes a multiplication with the elements. If you want to visualize think of a matrix of either row or columns i.e a single dimension when we multiplies we get an array of same shape but of lower or higher values, thus it helps in maximizing or minimizing the intensity of values.

This image might help you,

For details refer, https://www.youtube.com/watch?v=qVP574skyuM