Can an ML model choose between an arbitrary set of classes? I want to build a model that can take in X sets of information (e.g. you can think of it like X python dictionaries, all with the same fields) and choose one out of the X based on a bunch of examples I've fed it in the past (each training example consists of a variable-sized set of options, plus the set that should be chosen).
What are some ways I can formulate this type of problem? It seems in some sense like a classification problem, but the set of classes is not clear to me (especially because the number of inputs can vary between 1 and, say, 10).
Is this a regression problem, wherein I assign a score to every set and then choose the one with the top score? 
Are there analogous problems that other people have solved that you can point me to?
Thanks so much!
 A: You are asking how to deal with sets of inputs instead of vectors/matrices which is the much more usual case. I will be using Deep Sets by Zaheer et al. (https://arxiv.org/abs/1703.06114) as a jumping off point. I recommend giving it a read.
Since you are specifically looking to pick out a specific $x$, I agree with your intuition that you should output a score (or probability) for each input, with a target score of either 0 or 1 depending on whether it is the $x$ you want to extract.
The Deep Sets paper linked above recommends using the standard neural network formulation of $f_\Theta(x) = \sigma(\Theta x)$ where $\Theta \in \mathbb{R}^{m \times m}$ and $\sigma\colon \mathbb{R} \to \mathbb{R}$ is a non-linearity such as the sigmoid function. But, you restrict $\Theta$ such that all off-diagonal elements are the same, and all diagonal elements are also the same. I.e. $\Theta = \lambda \textbf{I} + \gamma (\textbf{11}^T)$ where $\textbf{I} \in \mathbb{R}^{m \times m}$ is the identity matrix and $\textbf{1} = [1, ..., 1]^T \in \mathbb{R}^m$. You can stack multiple layers of this form without losing permutation equivariance.
I think it may also be worth trying out a slight extension where you learn mutliple functions, one function $f(\{x_1, ..., x_m\})$ will learn a context vector $c$ that will contain collective information about your current set of objects. The second function $g$ will operate on an individual $x_i$ and the context vector: $g(x_i, c)$ and will output the score. Then only the function $f$ needs to be permutation invariant (not equivariant), which could be accomplished using Theorem 2 from the paper by decomposing the function into the form $f(X) = \rho \left( \sum_{x \in X} \phi(x) \right)$ where $X$ is your current set of inputs. Note that nothing fancy is going on here, we're simply taking advantage of the fact that the sum is a commutative operator so it doesn't care about ordering.
