I have binary medical data (6000,200) and I removed all duplicates (even though all records generated are unique data and of different people- they showed similar pattern and thus duplicate data came in dataset). Then I applied chi square test and discarded non-significant features. Then features reduced to 20 from 200 features...
So now I again checked for duplicates and found only 100 unique samples in the data set. So now data is of shape (100,20) Is it okay to remove such duplicates even though they belong to different people?