1
$\begingroup$

I have been given the task to classify some tweets per topic. I have done a classification based only on the per-document-per-topic probability with LDA. I have been suggested to use BTM instead, so I am working on that too. The dataset I work on is made of 3 columns:

  • the text,
  • the username, and
  • the day and hour the post was uploaded.

The problem is that during every classification I made or attempted, I only used the text column. I have been told I should use all three, but I have no idea about how to implement this, and I can't see why usernames should be relevant either for a topic classification. I need to mention that I do not have ground truth, so all techniques I can use are not supervised.

I would be extremely thankful if anyone could share his ideas with me or suggest a path to follow. Thank you in advance.

$\endgroup$
0
0
$\begingroup$

Recently I come cross following DataSkeptic podcast Face Mask Usage during the COVID-19 Pandemic https://dataskeptic.com/blog/episodes/2020/face-mask-sentiment-analysis (https://arxiv.org/abs/2011.00336)

In which data harvested from Twitter accounts are used. As far as I remember, they used date to analyse the trend over time (temporal aspect) and cross check /validate with other twitters. Username (name, surname, bio) is used to determine demographic .

If you haven't see the paper or listened to podcase, it might give you hints in your problem constraints.

$\endgroup$
0
$\begingroup$

Based on your description, the problem is more of a clustering rather than classification problem. The username matters because, well, people care about different things (a reporter is more likely to tweet about a political subject rather than say a chef.)

You can use bi-clustering ideas, simultaneously clustering tweets (represented by words, bigrams, etc.) and usernames. "Bipartite network clustering" or "co-clustering" are other good terms to search. Have a look at this paper

Some further thought has to go into it because the resulting say word-username matrix is very sparse. But this should at least give some new ideas.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.