I do research on educational games, and some of my current projects involve using data from BoardGameGeek (BGG) and VideoGameGeek (VGG) to examine relationships between design elements of games (i.e., "set in World War II", "involves rolling dice") and player ratings of those games (i.e., scores out of 10). Each of these design elements corresponds with a tag in the BGG or VGG system, so each element is essentially a dichotomous variable. A game has a 1 for every tag that's present in the database for it, and a 0 for every tag that isn't present.
There are dozens of these tags, so I want to use exploratory factor analysis (EFA) to come up with a manageable number of "genres" that capture patterns in game design. Consulting several sources, I understand that since I'm working with dichotomous variables, I ought to use polychoric correlations (tetrachoric, particularly here) instead of Pearson ones when coming up with my factors (there are also other options—like latent trait analysis—out there, but this is the one I'm exploring for now).
Out of curiosity, I came up with two sets of factors, one using Pearson correlations and the other using polychoric correlations (same number of factors each time). My problem is that the factors computed using Pearson correlations make a lot more sense and are easier to interpret than the factors computed using polychoric correlations. In other words, the "genres" from the first set of factors make intuitive sense and correspond with my understanding of how games are typically designed; that is not the case for the second set of factors.
On one hand, I want to make sure that I meet the assumptions of the tests that I'm using, even if that makes my results less pretty. On the other, I feel that part of the goal of factor analysis and (more broadly) model-building is to come up with something useful, and the more useful information is emerging when I'm "breaking the rules." Is the need for a useful model enough to outweigh violating the assumptions of this test? What exactly are the consequences of using Pearson correlations instead of polychoric ones?