What machine learning architectures can be used when dealing with variable input size? I have a machine learning problem where a single training example can be represented by an m by n matrix, where $m$ is fixed but n can vary. Essentially, a training example consists of n instances, each with $m$ features. The task is to perform binary classification. I want to find a ML model to deal with this data with the following properties:

*

*The architecture should be able to deal with the fact that n is not fixed.


*The predictions of the model should be invariant under permutations of the n columns.


*The architecture should be able to make use of all or almost all of the information encoded by the matrix, up to permutations of the columns.


*(Less important) Uses as few parameters as possible while satisfying the first three conditions
Does anyone have any suggestions for suitable models, particularly NN architectures?
The main idea I've had so far was to compress the matrix into a vector of length $m$ by finding the average value of each feature across the $n$ instances and feed that into a dense NN layer but that fails to satisfy property 3 since we will lose all information about which features appeared in the same instance.
 A: Here are some possible approaches:

*

*Use Deep Sets.

*Estimate a probability density for each set of your $n$ $m$-dimensional vectors (the columns) and take the parameters of this density as input for a model (thus, each matrix is mapped to a probability density).

*Pad the matrices with zero columns so they have all the same number of columns and then augment your dataset by adding for each matrix-label pair $(m, l)$ each column-permutation $(\pi_i(m), l)$ and then train an ordinary model on this new extended dataset.

A: Transformer neural networks seem like an obvious candidate. They are most famous for being used for text data, where of course the input size is variable and the order is fixed. However, note that transformers for text data had to add an extra component to deal with the ordering of the input (i.e. positional encodings), while a basic transformer without that would treat input as not having any particular order.
The other obvious option is to somehow summarize information across the multiple records into a fixed dimensional set of features, which probably would have to be done based on human expertise/judgment. Thereafter, a much wider variety of ML algorithms can be used straightforwardly. However, it may be challenging to find a set of features that does not loose too much information. E.g. it can be very challenging to deal with interactions of features (an example of this could be if an individual column with feature A being high and feature B being high + a second column with both low results in a high probability, but not when one has A high and B low + the other A low and B high).
