0
$\begingroup$

Please forgive my ignorance and lack of experience: I am asking this question seeking answer from the experts/experienced persons in the field.

I have a training dataset where each sample is a 3D cube (x,y,t) with grid points of (Nx = 256, Ny = 256, Nt = 200). The input consists of spatio-temporal evolution of a specific wave propagation. However, the input is also corrupted by other unwanted wave modes and noise components. I need to build a neural network (possibly convolutional auto-encoder or something similar) that can provide 3D output cube where the target wave propagation is mapped in space and time, effectively denoised of the unwanted wave modes and noises.

My question is: should I consider a 3D convolutional neural network for this task? Or, since each (x,y) frames are stored in a collection of a time series from t(1) to t(n) in the 3D cube, should I try a Recurrent 2D convolutional neural network for this job? Is there any advantage of trying one over the other?

Eventually I have to do this for 4D cube (x,y,z,t) with grid points of (Nx = 256, Ny = 256, Nz = 100, Nt = 200). In that case, would a Recurrent 3D convolutional neural network be better than 4D convolutional neural network?

If you have any literature in mind regarding this, please share. Thanks in advance.

$\endgroup$
0
$\begingroup$

The question to ask knowing how filters work is will looking at them all at once help? A filter is a nnd size and is placed at each nnd grid. It computes a dot product across the depth of the image. So in rgb case, to classify a region we want to look at all rgb channels in a single view as they provide complementary information about the object at that point. However in your case, you have to ask if the noise at the 100'th image influences noise at the first few images. The filters will necessarily try to form such links as they look through the depth.

3D filters go furthur, being of size nnd*p, they not only look through depth but also across the temporal blocks. In order to segment the square on the top right of the first image, you will be looking at the top right of all images across time. Try to visualize it and see what the dot product is considering. Ask if looking at all this information is needed to segment one region. If not, then your network will have a very hard time figuring the same out.

I think recurrent networks will work better.

Btw: As segmentation is poorly explained as compared to classification I made a video to provide some intuition. You may find it useful :https://youtu.be/NzY5IJodjek

Let me know if you have any more questions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.