why is the receptive field size not depend on the input image? Let say if I fixed the filter size as 3x3, then shouldn't an input of 128x128 image has a larger receptive field size than an image of 32x32?
In addition, what would be the affect of stride and padding on the receptive fields?
thank you.
 A: 
"Receptive fields are defined portion of space or spatial construct containing units that provide input to a set of units within a corresponding layer.
The receptive field is defined by the filter size of a layer within a convolution neural network."

https://towardsdatascience.com/understand-local-receptive-fields-in-convolutional-neural-networks-f26d700be16c
In other words, receptive fields are about the area (in pixels) covered by the convolutional filter(s), not the size of the input.
Padding changes the effective input size, so it can’t change the receptive field.
Stride changes how much area is covered by the convolution, so it increases receptive field.
A: To answer the second part of the question:

In addition, what would be the affect of stride and padding on the receptive fields?

As explained in Nikolas Adaloglou's blog on AI Summer Understanding the receptive field of deep convolutional networks, increasing the stride in a convolutional layer will increase the receptive field in subsequent convolutional layers. Padding doesn't change the receptive field.
For example, consider a network with two $3\times3$ convolutional layers. The first layer has a receptive field (RF) of $3\times3$ irrespective of the stride. The RF of the second layer changes with different strides used in the first layer. If the stride of the first layer is 1, then the RF of the second layer is $5\times5$. If the stride of the first layer is 2, then the RF of the second layer increases to $7\times7$ and if the stride of the first layer is 3, then the RF of the second layer is $9\times9$.
