I am attempting to analyse the categorical overlap of a <a href="https://raw.githubusercontent.com/iWinallS/pyUpSet/master/sample_data.csv">dataset</a> to ultimately ascertain the optimal way of categorising the data to minimise the amount of used categories to describe the dataset. 

**Strategy that I am trying to employ:** 
 1. map out all combination of "overlaps" with items and "independent" to identify items and count. ("independent" = items from a category not referred to by other categories) 
 2. out of the above combinations, identify duplicate data categories (ie. categories that is made up of 100% overlaps with other category/combination of full categories)
 3. output: unique number of these combinations, netting out duplicated categories will be the minimum amount of categories required to describe the data.
 4. itemised output: will be used to investigate further pruning of categorisation to identify small combinations that can potentially be rolled into a bigger category.

**Effort thus far:**

I am relatively new to R and python and have been doing a bulk of these in Excel. 
 - I have been able to derive a <a href="https://raw.githubusercontent.com/iWinallS/pyUpSet/master/output_matrix.csv">matrix</a> to analyse 1° of overlap (ie. #items overlapping per pair of categories) following the advise in <a href="https://stackoverflow.com/questions/32188560/category-overlap-analysis"> Category Overlap Analysis question </a>.
 - unpivoting the matrix to derive an <a href="https://raw.githubusercontent.com/iWinallS/pyUpSet/master/output_toFrmCount.csv">output</a> that can be used to analyse overlaps per pair of categories and identify duplication that exists for 1° overlaps. Through this <a href="https://github.com/iWinallS/pyUpSet/blob/master/analysis.xlsx">analysis</a> I am able to identify 300 duplicated 1° categorical overlaps which can be resolved to 118 unique categories.

short coming of the method above:
 - inability to identify item count for higher order of overlaps
 - does not produce an itemised output for each combination of "overlaps"/"independent"

I have also attempted to use visualisations techniques to achieve the outcome, but to no avail:

 - Chord Diagram: there is 1200+ categories, the limit to chord diagram is 400+
 - <a href="https://vcg.github.io/upset/">UpSet</a> with this <a href="https://raw.githubusercontent.com/iWinallS/pyUpSet/master/output.json">json</a>: but can't to configure it to derive the insights needed.



**Question:**
 1. is there a scientific name of this type of analysis (potentially use this to update the title of this question and to do further research on the best method to do this)
 2. is there a better/easier (more scalable, efficient and effective) way to do this than what I am doing?


----------

**EDIT**
Example to better illustrate what I am having difficulties attempting to achieve.

*Data snippet*

    Category, ItemCode
    G0617,5410.001
    G0617,5410.006
    G0617,5410.903
    ...
    G0080,5410.001
    ...
    G0419,5410.001
    ...
    G0532,5410.001
    G0532,5410.903
    ...
    G0616,5410.006
    ...
    G0659,5410.001
    G0659,5410.903
    ...
    G0846,5410.001
    ...
    Gtest,5410.903
    Gtest,5410.006

[![enter image description here][1]][1]

*Ideal output*

    category|equivalent categories|subsumed categories|independent item_code|duplicated category
    G0080|||5410.001|
    G0419|G0080|||TRUE
    G0532||G0080|5410.903|
    G0616|||5410.006|
    G0617|G0532,G0616;G0080,Gtest;G0532,Gtest|G0080,G0419,G0532,G0616,G0659,G0846||
    G0659|G0532|G0080|5410.903|TRUE
    G0846|G0080|||TRUE
    Gtest||G0616|5410.903|

[![Ideal Output in a worksheet][2]][2]

Duplicated category will be biased towards category ID with smaller #. 
(ie. Where same categories are exactly the same, the category with higher ID will be marked as a duplicate of the category with the smallest ID. This example, G0419 and G0846 are both marked as duplicates of G0080)


  [1]: https://i.sstatic.net/sweUk.png
  [2]: https://i.sstatic.net/agvE1.png