I want to perform a linear regression model on a dataset with some bestseller books, the dataset contains 550 the bestseller books - I want to create a lm() model where I predict the variables that are most influential when it comes to becoming a bestseller, however, there is no obvious Y variable in the dataset since all the books are bestsellers. How would you solve that? Would you add a Y variable, and if yes, which? Or is there another solution?

  • 5
    $\begingroup$ You cannot figure out why bestseller books are bestsellers... using only bestseller books, there is nothing to compare to. You would need to have preferably an equal amount of both type of books in your data, bestsellers and non-bestsellers, and then your y could be 1 or 0. $\endgroup$ Dec 11, 2020 at 7:53
  • $\begingroup$ @user2974951 Since most books are not bestsellers, I would actually want to have more of them in the dataset. Preferably, the dataset should be a random sample (of sufficient size). $\endgroup$
    – Roland
    Dec 11, 2020 at 8:47
  • 1
    $\begingroup$ Are you able to share a sample of the data eg using dput in R or host it online? Would make it easier to visualise. Even in a bestseller dataset there may be a disparity in how many they've sold $\endgroup$ Dec 11, 2020 at 8:50

1 Answer 1


@user2974951 addresses the problem here. In order to determine which variables best predict bestselling books, you'd need to better define your population under study, which would have to include best-selling books in addition to non-best-selling books. For example, you may be able to obtain a list of books published and made available to the library of Congress in a given year. You would then need to cross-reference that list with, say the New York Times Best Seller list (you'll need to define "best-selling" more carefully too because there are many "best-seller" lists). Once you have identified best sellers, one approach is to use logistic regression. Using this approach, you could code a variable, say, best_seller, as 1 if the book is a best-seller and 0 otherwise.

Once your dataset is properly coded and includes observations from both non-best sellers and best-sellers there are a number of approaches for determining the "most influential" variables in predicting best-sellers. You could use methods like simple regression with standardized coefficients, or better yet logistic regression. There are a number of other methods in machine learning that may be even better suited for this task such as the xgboost algorithms like adaboost. What you are after is essentially called variable importance or feature importance in the machine learning communities. If you search Cross-Validated on these terms, you'll find a number of methods that you can try out.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.