How to handle 2 input representations ("channels") of a data that is associated with the exact same label?

Question

I have a data sets of matrices with non negative values. For each learning instant I have 2 representations of the input data which corresponds to the same label.

Namely for each matrix label $\mathbf{Y}_{N\times N}$ I have 2 labels $\mathbf{X}_{N \times N}^1$ and $\mathbf{X}_{N \times N}^2$.

The output/label matrix is matrix of 0's and 1's.

Since $\mathbf{X}^1$ and $\mathbf{X}^2$ are matrices in different representations of the input signal I would like to use both as inputs. The thing is that the average value of $\mathbf{X}^2$ is significantly larger then $\mathbf{X}^1$, so I am not sure that feeding a Neural Network with 2 channels is a good idea.

1. Do you have a suggestion what should I do, beside training 2 separate networks and having some voting mechanism?
2. Can I use some multichannel network, although the scale of each input matrix is different.
3. What if $\mathbf{X}^1$ $\mathbf{X}^2$ are quite sparse (~20% aren't non zero elements)?

Answer 1

Since we' are talking about images I would assume that each image shows the same but in a different representation. Therefore, it makes sense to feed the input as one image with different channels. In this way the network is able to find corresponding features based on locality in your picture.

So if we disregard batch_dimension your input shape will look like this: (N, N, 2).

Can I use some multichannel network, although the scale of each input matrix is different.

Yes although I think the upper approach would be the better one. If you are worried about the scale of your input then just rescale it.

What if X1 X2 are quite sparse (~20% aren't non zero elements)?

Since we're talking about images, I think Convolutional Layers should be a key part of your Network. Those layers are using weight sharing, which makes them very robust against sparse inputs. Have a look at the MNIST Dataset, which is often used as a basic example for Convolutional Networks. The images there are quite sparse too.