What is a figure of the learned function in binary classification called? Correct me if I'm wrong. When we talk about overfitting and underfitting we talk about how the output function made by the neural network is fitted in relation to the data. 
What is this function called? Is this just a visual aid to help understand the concept of over- and underfitting, or is it possible to actually plot this function for simple problems with 3 or less dimensions?
Image from Wikipedias overfitting article:
 
Can I plot these lines? What are these lines called?
 A: The figure shows a decision boundary for a binary prediction problem. It is a two-dimensional representation of the learned function and shows how the model would decide between the two classes in these two dimensions. 
If you have more than two predictors, such a figure is less informative and you may instead want to plot the ROC curve of the model. You could technically still make a similar graph using e.g. PCA or MDS to plot the function along two axes representing the largest variability, but you should be careful how you interpret that.

As you already mentioned, one particular use of a figure of the decision boundary is to visualize over- and underfitting. Another potential use is to visualize the effect of moving the threshold for either decision. For example, in logistic regression you could choose to classify the outcomes as $A:y \leq 0.5$ and $B:y > 0.5$, but you can choose different thresholds as well.
That said, both of these purposes can be fulfilled by better alternatives (e.g. out-of-sample performance estimates) and I think this type of figure is especially useful for didactic purposes.
