Given a high dimensional feature vector x in $R^D$, I want to map it to an L bit vector, $L << D$,
z = h(x) in $\{0,1\}^L $
using a function h while preserving the neighbors of x in the binary space.
My problem is closely related to the one asked here Activation Functions
and based on https://mediatum.ub.tum.de/doc/680212/680212.pdf
The activation function f()
is used as a unique code generator. What learning algorithm should I use ? f() is of the form :
https://en.wikipedia.org/wiki/Piecewise_linear_function
Please correct me where wrong. If I want to apply this piece-wise linear (PWL) function as a classfier, then does that mean that the function forms an activation function in neural network or can there be other mechanisms where this can be applied? Any small simple example to show what will be the input and what will be the output. What is the role of neural network in this problem? I am having difficulty in understanding what I need to do so that the PWL function can be used as the classifier function and the role of neural networks. As an example :
Let:
$x_1 = 5.1,3.5,4.9,-1.40,-0.2,3.2$ ; $y_1 = 1,1,1,0$
The feature dimension in this case is 6 and the class labels is a vector of L = 4 bits. Each example is assigned a unique binary string of length L. We can Refer to these strings as codewords. Then L binary functions are learned, one for each bit position in these binary strings. During training, the desired outputs of these L binary functions are specified by the codeword. With artificial neural net w orks, these L functions can be implemented by the L output units of a single net work. Now, I want to use the PWL function as the activation function. How can I do this? Any example to help clear the concepts will be very useful.