As an English major with no traditional training in statistics, I am having a very rough time with this, so any help would be greatly appreciated.

My problem is that only 849 books out of my 6360 book dataset have full texts on gutenberg.org, and those that do are skewed towards popularity. Here is the dataset that I've collected (data taken from At the Circulating Library and Goodreads), and here is a boxplot of how skewed the presence of full text files of the novels is towards popularity (Ys have Gutenberg pages, Ns don't).

I would like to eventually do some hypothesis driven text mining on the text of the novels, and make a correlation plot between the number of Goodreads reviews and the data from the novels in order to see if there is a connection between certain features of the texts and present day popularity. I know that I will need to normalize the data somehow so that the sample (containing novels that have Gutenberg IDs) is representative of the entire literary field of the period (the full dataset), and that the Goodreads data should be the "normalizer", but I am completely clueless about how to proceed.

All I need is a gentle push in the right direction, and I'll take it from there. Thanks in advance!

  • 1
    $\begingroup$ I suppose this could be seen as a missing data problem. You have observations where you have one variable but not the other. It might be possible to make some progress using inverse probability weighting but if you have zero background in statistics that might be a leap too far. $\endgroup$ – mdewey Jun 20 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.