Conceptual diffiulty in understanding multiclass classification Given a high dimensional feature vector x in $R^D$, I want to map it to an L bit vector, $L << D$,
z = h(x) in $\{0,1\}^L $
using a function h while preserving the neighbors of x in the binary space.  
My problem is closely related to the one asked here Activation Functions
and based on https://mediatum.ub.tum.de/doc/680212/680212.pdf
The activation function f() is used as a unique code generator. What learning algorithm should I use ? f()  is of the form :
https://en.wikipedia.org/wiki/Piecewise_linear_function
Please correct me where wrong. If I want to apply this piece-wise linear (PWL) function as a classfier, then does that mean that the function forms an activation function in neural network or can there be other mechanisms where this can be applied? Any small simple example to show what will be the input and what will be the output. What is the role of neural network in this problem? I am having difficulty in understanding what I need to do so that the PWL function can be used as the classifier function and the role of neural networks.
As an example :  
Let:
$x_1 =   5.1,3.5,4.9,-1.40,-0.2,3.2$ ; $y_1 = 1,1,1,0$
The feature dimension in this case is 6 and the class labels is a vector of L = 4 bits. Each
example
is
assigned
a
unique
binary
string
of
length
L. We can Refer
to
these
strings
as
codewords.
Then
L
binary
functions
are
learned,
one
for
each
bit
position
in
these
binary
strings.
During
training,
the
desired
outputs
of
these
L
binary
functions
are
specified by
the
codeword.
With
artificial
neural
net
w
orks,
these
L
functions
can be
implemented
by
the
L
output
units
of
a
single
net
work. Now, I want to use the PWL function as the activation function. How can I do this? Any example to help clear the concepts will be very useful.
 A: Classification :
The introduction of the paper you quoted contains your answer I believe :

This problem is split in many two-class classification problems

To classify something into L classes {1,2,..,L}, you can cut your problem in L classification sub-problems :


*

*{1}/{2,3,..,L} 

*{2}/{1,3,..,L} 

*...

*{L}/{1,2,...,L-1}


Each of these sub problems can now be easily solved by a neural networks giving you L binary coordinates.
If your neural-networks are linear (perceptron) then you can also visualise the results with a PWL (as in figure 3 of the paper you quoted).
For such classification task, the current approach is to write a single network using a sofmax output layer.
Dimensionality reduction :

using a function h while preserving the neighbors of x in the binary space.

If you want to preserve the neighbours then that looks like dimensionality reduction rather than classification. 
An auto-encoder with an output layer of size L made of strong activation functions would take your data and express them in {0,1}^L (each bit being the binarised output of it's corresponding activation function).
(If you don't especially need a binary output then other methods such as PCA and t-SNE might bring better results (preserve the neighbours) depending on the exact nature of your problem.)
