2
$\begingroup$

So, I have survey responses from users. Just to make it clear, if you select an issue like Poor UI then you are prompted with 4-5 specific issues about the UI to select from. Poor UI is the main variable and then we have 4-5 sub-issues related to the poor UI. For example,

Q1. What do you have a problem with?

  1. Poor UI - Select Yes or NO. If yes, you are shown specific issues about Poor UI to select from.

Out of 9 variables, the user can select 3 only, and then they are shown sub-issues related to each of the selected issues.

I am new to data visualisation and transformation so I was thinking about using a dimension-reduction technique like PCA or MFA but I would like to know if there's anything else that I can do to have better visualisation and analysis.

I also want to test the variables across the different user segments. I need to know which user segment faces statistically significant issues as compared to others.

I will be using R for it.

Thank you.

$\endgroup$
6
  • $\begingroup$ Hi and welcome to CV! Could you tell us something more about the data? I have no idea what a close-ended response is. $\endgroup$
    – utobi
    Commented Nov 14, 2023 at 13:54
  • $\begingroup$ Do you mean you have 9 questions or statements, each of which is answered using a 5-point Likert-style scale (e.g. from totally disagree to totally agree)? $\endgroup$
    – Sointu
    Commented Nov 14, 2023 at 14:13
  • $\begingroup$ Edited with more information. $\endgroup$
    – doodle2611
    Commented Nov 14, 2023 at 14:39
  • 1
    $\begingroup$ So, for each person, you have info on which 3 issues they had a problem with, and then, for each of the 3 issues, you have info about which sub-issues they selected? If so, this type of data doesn't really "bend" to statistical data reduction techniques. Maybe you could visualize the data by just by graphing the frequencies of the problem issues and subissues? $\endgroup$
    – Sointu
    Commented Nov 14, 2023 at 14:55
  • $\begingroup$ Hi! You say "I was thinking about using a dimension-reduction technique like PCA or MFA", but you don't mention to what end. My answer below assumes you're interested in visualizing the global repartition of issues and subissues; if it's not the case, then you should mention what kind of information you are trying to get from your data. $\endgroup$
    – J-J-J
    Commented Nov 14, 2023 at 19:40

1 Answer 1

3
$\begingroup$

If you want to identify which issues and sub-issues tend to co-occur in your dataset, multiple correspondance analysis may be an adequate tool here, for exploratory purposes. Treating "No" as a sub-issue in your data might be useful. Here is an example, with a made up dataset where users can choose among 3 main issues, and 3 related sub-issues:

Multiple correspondence analysis graph, showing which issues and subissues are related

For interpretation, see How do I interpret a plot of variables created in FactoMineR? , but for example we see that "sub-issue 3" from issue 1 and issue 2 are close together (upper left quadrant), and "sub-issue 1" of issue 2 and "sub-issue 2" of issue 3 seem to repel each other (top and bottom of the graph). As you have many sub-categories, the graph will probably be hard to read, so you may want to merge some categories together, or exclude some of them from the analysis altogether. What to do depends on your goal.

As you mention segments of users, you can throw in some variables related to your users (gender, location, etc.), to see how their values relate to specific issues (if you have some variables that are not categorical, a relevant extension of MCA to use here would be factor analysis of mixed data). You could include these variables as active or supplementary variables, depending on how you want to conduct your exploration.

Finally, note the graph can get tricky to analyze co-occurrences of issues, if a lot of users tend to pick only one single main issue. In this case, the occurences of "No" will dominate the graph, which could make it uninformative. A solution to this problem may be to restrict the MCA graph to the subset of users who have complained about more than one issue.

There are multiple R packages offering MCA or FAMD. I used the package FactoMineR to generate the graph above, but alternatives include GDATools or PCAmixdata. Other programming languages or statistical software generally offer this kind of feature, e.g. in Python there's the nice Prince library doing this kind of thing.


On the other hand, if you want to see the repartition of complaints among the main issues and sub-issues and how their flow is directed, you could use a Sankey diagram.

Here's an example showing the repartition of the US coronavirus stimulus bill, in terms of money, in many categories and subcategories (the graph is a courtesy of SevenandForty, license CC BY-SA):

Sankey diagram titled "Where does the money go in the US Senate's $2T coronavirus stimulus bill?". It shows the repartition of CARES Act relief amounts. On the left, one main category "CARES Act" consisting of 2,000 billions dollars, branches into 6 sub-categories, 2 of which got about 500 billions dollars, 2 others between 300 and 400 billions, another one 153 billions, and the last subcategory 26 billions. These 6 sub-categories then branches into smallest sub-categories (about 30).

However, in your situation, the problem is that you might end up with many nodes and links, making the graph hard to read.

Possible strategies to avoid this problem include:

  • hiding or showing some nodes of your choice, given a certain criteria of interest (e.g. hiding all nodes under or above a certain number of occurrences, or hiding nodes related to systemic problems that would require too much investment to solve on the short run). In this case, you should add a reminder somewhere on the graph, mentioning that some categories have been hidden.

  • merging some categories or sub-categories together to make the graph more readable. You could use a quantitative or qualitative criteria for this purpose, for instance:

    • merging categories or sub-categories with low counts in an "Other issues" category;
    • merging categories that respondents often choose simulteanously. For example, if respondents always choose "Poor UI" and "Poor accessibility" simulteanously, then you could merge them in a "Poor UI & accessibility" category;
    • merging categories that, from your domain knowledge, you identified as extremely similar (even if users do not seem to choose them simulteanously).

Note that it might be tricky to merge two parent categories together, because then you have to decide what to do with their children (Merge all of them too? Keeping all of them distinct? Keeping only some of them distinct, and merge some others?).

Another drawback to this kind of visualization is that by default it won't show issues or sub-issues that are never a cause for user complaints, unless you tweak your data by adding a really small count to these issues - but this could make your graph somehow misleading. If you think you need to do that, a (not very satisfying) solution to mitigate this problem could be to add "0" next to the label of these categories, and giving further explanations in the footnotes. Otherwise, creating a simple, separate list or table of issues that never occur could be a solution. Maybe there are other solutions to this problem, but it should probably be a separate question.


All these decisions to take (which nodes to hide or to merge, etc.) require knowing the eventual purpose of the visualization, which you don't mention in your question.

Some R packages offering Sankey diagrams generation are networkd3, ggalluvial, and plotly. There are probably others. From experience, it may be sometimes a challenge to make this kind of graph nice to read.

$\endgroup$
5
  • $\begingroup$ Merging won’t be possible but Sankey can be used to show the overall visualisation. The main goal is to fit or try to fit all of this in single graph and then zoom into the most prevailing categories. I also want to run statistical tests if any possible between the variables. $\endgroup$
    – doodle2611
    Commented Nov 15, 2023 at 0:10
  • $\begingroup$ @raothar98 If you're interested in the most prevailing categories, then I'm not sure what prevents you from merging non-prevailing categories in a category "Other issues", to simplify the graph. But obviously you know your situation and data better than me. $\endgroup$
    – J-J-J
    Commented Nov 15, 2023 at 6:02
  • $\begingroup$ @raothar98 About statistical tests: You should edit your question to mention that you want to run statistical tests. In this case, you should also mention what you want to test, because it's a bit difficult to suggest something without knowing its purpose. For example, do you want to test if issues are (un)evenly distributed? Or something else? If you have some data about your users, you could also run some tests to see if some categories of users are more likely to complain about specific issues, but possible tests depend on the kind of data you have. $\endgroup$
    – J-J-J
    Commented Nov 15, 2023 at 6:05
  • $\begingroup$ I have updated it. Sorry, i am new this forum so not sure how it works. Thanks. I have data for different user segments and want to see if we can test if a user segment faces more issues as compared to another and if it is statistically significant. $\endgroup$
    – doodle2611
    Commented Nov 15, 2023 at 7:43
  • $\begingroup$ @doodle2611 Absolutety no problem, this is very common to ask for clarification on this website. I updated my answer at the same time you updated your question, and I added some suggestion for using multiple correspondence analysis, which in fact might be appropriate here. (This does not answer your question about statistical tests, but this might be helpful, still). $\endgroup$
    – J-J-J
    Commented Nov 15, 2023 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.