1
$\begingroup$

I have to build a logistic regression classifier to predict $\mathbf{y}$ given $\mathbf{x}$ where $\mathbf{x} \in \Re^{n}$ is an image and $\mathbf{y} \in \Re^{m}$ is a binary attribute vector (of $m$ attributes) with each $y_{i} \in [0,1]$.

The problem statement is that we have to build $m$ independent logistic regression classifiers to learn the mapping $\mathbf{x}\rightarrow y_{i}$ for $i = 1,2,..m$. I don't think this is a multi-label classification problem since the $m$ classifiers are learnt independently.

Kindly help me with the best strategy to solve the above problem. I can use sklearn and keras libraries only. There is no need for code; just let me know the best way to solve the above problem.

$\endgroup$
1
$\begingroup$

If we have already decided to use $m$ separate models, we train each model $i$ with data $(\mathbf{x}, y_i)$. Then, we feed new data $\mathbf{x}$ to $m$ models and concatenate their outputs as $\mathbf{y}'=(y'_1,..,y'_m).$

If we model this problem with $m$ separate models, we are assuming that given $\mathbf{x}$, $y_i$ is independent of $y_j$ for all $i$ and $j$. Nothing is wrong with this assumption and it could work. However, if this independence assumption does not work in practice, we could generally use a neural network with $n$ inputs, $m$ outputs, and loss function $\parallel \mathbf{y} - \mathbf{y'} \parallel^2$ for output $\mathbf{y'}$ and true attribute $\mathbf{y}$. We can use sigmoid function for the last layer to have $y'_i \in (0, 1)$. This way there is no independence assumption, and model $y'_i$ has shared parameters with $y'_j$. Of course, we can experiment with many design choices to see what network structure works best.

I don't think this is a multi-label classification problem since the $m$ classifiers are learnt independently.

If multiple $y_i$'s can be $1$ at the same time, you are right, it cannot be cast to a multi-class problem, since only one $y_i$ in $\mathbf{y}$ should be $1$ at a time, or at least $\sum_i y_i = constant$.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.