Approximating a function of a cloud of (x,y) points My objective is to approximate an (unknown) non-linear function whose:


*

*input is an unordered collection of an arbitrary number of (x,y) coordinates (imagine a cloud of, say, between 1 and 1000 points in two-dimensional space)

*output is one number 


I have example inputs and corresponding outputs produced by a numerical model that is very expensive to run. (Hence the motivation to approximate the function, rather than calculating it exactly every time.) 
Details of the model: a Computational Fluid Dynamics simulation, where the input points represent (identical) obstacles, of which there can be an arbitrary number, and output is the resulting time-averaged velocity deficit at the origin. The origin (output point) is upstream of the obstacles, so the function is fairly smooth. 
My thought was to design a neural network that could be trained to approximate this function. But I can't figure out what kind of neural network, if any, would be suitable for the job. 


*

*Feed-forward networks require a fixed number of inputs (whereas I have an arbitrary number of (x,y) points)

*Recurrent neural networks can handle arbitrary-length inputs, but inherently attach significance to the order of these inputs and will give different outputs for different orders (whereas the "order" of the points in my input collection isn't defined and therefore shouldn't influence the output) 


What would be a suitable high-level neural network design?
Or are neural networks just not the right tool for the job?
 A: I can't speak to the feasibility any strategy in the context of CFD, but have some general concerns about how complex the function might be (from the comments, I'm curious to hear DeltaIV's ideas). Instead, I'd just like to think about the regression/function approximation problem you raise: how can we learn a function whose input consists of a point cloud containing a variable number of points?
Distance metrics and kernel functions
One possible approach is to use a method based on distances or kernel functions, which measure the similarity between two point clouds. A suitably defined distance metric or kernel function can deal with a variable number of points in each cloud, and can be made invariant to their order. Defining this function is a crucial choice, and can be highly application specific. In this case, you may have to define a custom function. Intuitively, it should be defined such that, if two point clouds are considered similar, then their respective outputs should also be similar (the converse need not be true).
For example, some form of edit distance might be appropriate. Edit distances can be defined on many objects, such as strings, graphs, and sets of points. Here, you'd define a set of operations for transforming the point clouds, such as moving a point, deleting it, or inserting a new point. You'd also define the 'cost' of each operation. For example, the cost of moving a point would increase with the distance moved. Given two point clouds $x_1$ and $x_2$, the edit distance measures the minimum cost of transforming the point clouds such that each point in $x_1$ exactly matches a partner in $x_2$. Notice that $x_1$ and $x_2$ need not contain the same number of points, and the distance is invariant to the order of the points. A downside of edit distances is that they can be computationally expensive.
A kernel function on point clouds could also be defined. One way to do this is based on a distance metric (but this isn't necessary). For example, radial basis function (RBF) kernels are typically defined on vector spaces, using Euclidean distance. But, we could instead define one on variably-sized point clouds. Given a distance metric $D$ on point clouds and kernel bandwidth $\sigma$, the RBF kernel is:
$$K(x_1, x_2) = \exp \left (
  -\frac{D(x_1, x_2)^2}{2 \sigma^2}
\right )$$
Learning a function
The goal is to learn a function mapping variably-sized point clouds to real number outputs, based on a set of examples. Given a suitable distance metric or kernel function on point clouds (as above), this problem can be solved using standard regression methods.
As a particularly simple example, given a distance metric, one could use k-nearest neighbors regression. Given a kernel function, kernel regression or kernelized support vector regression could be used. Alternatively, one could use an RBF network, which is a type of neural net. There should be various tricks to reduce the computational load in each case, particularly if distance computations are expensive.
