2
$\begingroup$

Say I'm using Top2Vec as a topic model to capture the top 10 salient topics across documents. I have an array that contains the documents of the corpus. Initially, there are not enough documents to run the Top2Vec. There sample size is enough to run LDA, though, and get usable results. If next I multiply the array by, say, a factor of 10, there would then be enough documents to run Top2Vec. If I were to multiply the array of documents by 100 or 1000 instead, would the results always be the same assuming each original document is repeated proportionally (and identically) in the resulting array? Is scaling up this way trivial in terms of results, or would it distort results in some way? This is only for comparing the results of both approaches.

Edit:

I realize that multiplying the array of documents will not generate any new information, but the model won't even run without expanding the array somehow. I just need to get a sense of what the results would look like if Top2Vec were run. This is for the purposes of comparison to another model that already does generate meaningful results on the unaltered sample size, without any augmentation. The combined length is already 200,000 words without multiplying of the set. The question is whether such a comparison would be fair. Since no new information is generated by multiplying the set, it seems to me that augmenting just to get Top2Vec to run should be fine (it's not like results would change anyway whether it was multiplied by a factor of 10 or 100 since no new info is being generated, right?). I'm trying to make sure I'm not mistaken in that thinking, for the purposes of comparison. Even if the quality of Top2Vec's results is low on that amount of raw information, that would still be okay to show in order to demonstrate why the first model may be more suitable. Am I missing something?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Well, you can try out if you get a result (I don't know Top2Vec), however the problem really is the limited amount of information in your data, and what you propose doesn't add information. So even if you get a result, chances are this will be of low quality (or let's say the quality that your limited amount of documents will allow). $\endgroup$
    – Christian Hennig
    Commented Jan 20 at 11:00
  • $\begingroup$ Out of interest, how many documents do you actually have? $\endgroup$
    – Eoin
    Commented Jan 22 at 16:50

1 Answer 1

Reset to default
5
+100
$\begingroup$

Definitely not.

Initially, there are not enough documents to run the topic model.

This means that the number of documents you have does not provide enough information to generate a meaningful topic model. Duplicating documents to make the list of documents longer doesn't add any information, so does not solve the problem. In real life, a library with 10 copies of the same 5 books doesn't contain any more information than a library with 1 copy each.

The more interesting question is - why are you trying to run a topic model at all? If you only have a small number of documents, does it make more sense to just generate summaries of each (e.g. with GPT), rather than trying to force them into clusters?

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the reply. Of course, no new information will be added and the goal, believe it or not, isn't to get a usable result. I was asked to compare the results of Top2Vec on the sample with another model that can generate meaningful results on the small sample size. Even if the results are not usable from Top2Vec, I need to know if multiplying the sample set just to get it to run is bad practice, if only to be able to show what the results would look like. Note that Top2Vec will not even run without multiplying the array by at least a factor of 10. $\endgroup$
    – NominalSystems
    Commented Jan 22 at 4:17
  • 1
    $\begingroup$ Neither method is viable at this sample size, so you're truly better off with manual analysis. The value of statistical methods is characterizing uncertainty and aggregating over unreasonably large data. The error bars here would be so huge as to be uninformative, and this isn't so huge that you need computation anyway. (Look up "small-N studies".) Re: Top2Vec, just because it doesn't give uncertainty estimates doesn't make it better: you just don't know how uncertain it is! Using tools wrong to torture a number (or topic list) out of them isn't something anyone should ask of you. $\endgroup$
    – Arya McCarthy
    Commented Jan 22 at 4:32
  • $\begingroup$ Thanks. If it helps to know, the first method seems viable on the sample size (which already has over 200,000 words in combined length unaltered). The first method yields very usable and meaningful results. Even if Top2Vec doesn't yield meaningful results, that would be okay to show. I just need to know if multiplying the corpus to get Top2Vec to even run (it won't without any augmentation) makes the comparison meaningless $\endgroup$
    – NominalSystems
    Commented Jan 22 at 4:40
  • $\begingroup$ P.S even if it's just to show why Top2Vec is not suitable $\endgroup$
    – NominalSystems
    Commented Jan 22 at 5:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.