# Is regression possible for unstructured text?

Let's assume I have data consisting of unstructured text and some features rows, with numeric values (red, yellow, green, blue are labels used in some psychometric systems):

                   Text                     Red   Yellow   Green   Blue
----------------------------------------- ----- -------- ------- ------
Eine junge Frau aus L.A. hat über [...]   120       98      78    150
Hamburg hat es gezeigt: Die Gründe[...]   110      150      29    111


The goal is: having new input text (by user), predict the values for red, yellow, green and blue.

Is that even possible? Are there any algorithms that would be suitable for such task?

Another data is formatted in slightly other way, where label is one of four colors:

                   Text                     Label
----------------------------------------- -------
Eine junge Frau aus L.A. hat über [...]   Red
Hamburg hat es gezeigt: Die Gründe[...]   Blue


But the task is similar: Given some text I must provide information about all of the colors, f.i. red: 90%, yellow: 87%, green: 40%, blue: 20% or anything that would give some continuous result. Again, is that possible, and how should I approach this?

EDIT: I already have a model that does classification, but the stakeholders need a numeric representation for each label/color.

• You don't mention them but LDA (latent dirichlet allocation) models seem like an appropriate approach. David Blei has written some of the earliest and best papers on this class of models. Here's a link to an introductory video lecture he gave about them ... videolectures.net/mlss09uk_blei_tm Jul 27 '17 at 12:27

## 1 Answer

Yes.

The second format fits a straight-forward classification problem.
We have multiple classes(Red/Yellow/Green/Blue) and text based features.
There are two main components needed in order to creat a model that can predict class based on text.
1) Transforming the text into numeric representation(vectoring/space embedding).
Tf-Idf is a common way accomplishing this, and word2vec has a powerful API with a lot of text features extraction capabilities.
2) Fitting a classifier(decision tree for example) to the transformed text based on the given labels.(preferably one that support probabilities assignment)

There are many ways accomplishing these two tasks and if you know Python - Sklearn got you covered.

For further reading look into document classification.

According to your comment - if you do wish to predict multiple numeric values based on the text features them i would use the following logic.

If multiple predicted variables are independent of each other - use multiple regressions, one for each color.
If they are dependent - find the color that you can predict best, and use this model prediction(along with other text based features) to predict next model color.
If your first model preforms well, then you wont be creating too much noise when moving to the second(and then to the third, forth.. )

If your first model performance is bad, you can try clustering the colors space(4 dimensional) and treating the clustered space as a classification problem. If you decide following this line, you need to make sure that the cluster sizes meets you prediction accuracy demand.
Say you predict its cluster A, it could mean a lot of different color values, depends on cluster size.

I had some success with the hierarchical prediction(feeding one model prediction as a feature to the next). But the setup was somewhat different.

Hope this helps.

• Thank you for your reply. I have done already transformations with CountVectorizer (Tf-Idf didn't bring any improvements), and trained a LogisticRegression model. Yes, it gives probabilities with predict_proba, but this is still a classification task, where I was asked how to predict a number (regression task) for each category. Jul 27 '17 at 11:02
• then it is not a classification task anymore but a regression task. Jul 27 '17 at 11:47
• Thanks, would it be possible that you expand slightly what you wrote here: "you can try clustering the colors space(4 dimensional) and treating the clustered space as a classification problem."? Jul 27 '17 at 13:24