Topic modeling for regression

Question

Is there a way to influence the way topics are created with topic modelling in the sense that the topics also reflect their influence on the target variable of a machine learning problem?

I have a regression problem, where I try to predict the selling price of houses. One feature is a text-description of the house. It is my first project including NLP (natural language processing). From my research so far I thought it would be a good idea to try topic modelling. So a description like "Wonderful view from the balcony; cosy kitchen" is translated into a vector (0.2, 0, 0.3, 0.1, 0.4) meaning that this description belongs to topic 1 with 20%, to topic 2 with 0%, to topic3 with 30% and so on.

So far I tried topic modeling with LDA (Latent Dirichlet Allocation). The resulting topics make sense, but they have no connection to the house prices. When I calculate the dominant topic for each description and plot the distribution of the house prices for each topic, there's not much difference. It makes sense since LDA is unsupervised. But is there a way to include the house prices in the topic-learning-process?

My goal is to have topics which have a strong correlation with the house price. So topic1 may be interpreted as "very low price" and could contain words like "shabby, broken, small" and so on.

Answer 1

What you want is a supervised topic modelling.

I think the first paper is Supervised topic models by Blei and McAuliffe. (free access)

There are others papers on this topic that Scholar will find. There is also "semi-supervised" topic modelling, which I dont think it is what you want.

The problem I see is that as far as I remember, usually supervised topic modelling refers to a target class and not a target numerical value - that is, it is used in classification and not regression. If that is true, and there is no published research on supervised topic modelling for regression, that you will have to split your numeric target variable (house price?) into categories.