Assuming your dataset should be a random sample drawn from your population, it depends on what you define as a duplicate. I see two scenarios:
- "Duplicate" defined as two (or more) distinct observations that happen to have the exact same features/values.
In this case, you should keep them, as it's just how your population of interest is. You immediately see why if you have a dataset of let's say 2,000 observations, but only 2 features with 2 levels ("gender: man or woman" and "Exam results: pass or fail"). If you drop all the observations that are identical, you'll end up at best with 4 observations: 1 man who passed the exam, 1 man who failed, 1 woman who passed, and 1 woman who failed. So in this case, you would just be deleting useful information, and you'll be unable to infer anything about your population of interest.
- "Duplicate" defined as the same observation erroneously recorded twice or more
On the other hand, if you define a duplicate as the same observation incorrectly recorded multiple times, ideally you should drop them, because at some point something has been interfering with the sample random draw. In particular, it may be indicative of something going wrong with the data collection process, like a software bug, some typing error, or even fraud from respondents or interviewers, see Kuriakose, N., & Robbins, M. (2016). Don't get duped: Fraud through duplication in public opinion surveys. Statistical Journal of the IAOS, 32(3), 283-291.
This kind of duplicates may seriously bias estimates, see Sarracino, F., & Mikucka, M. (2017, April). Bias and efficiency loss in regression estimates due to duplicated observations: a Monte Carlo simulation. In Survey Research Methods (Vol. 11, No. 1, pp. 17-44). So it may be a good idea to not ignore them if you know they shouldn't be there.
In some situations, it might not be obvious if duplicates (or near duplicates) fall under the first or second category, in particular when you've not been involved in the study design and data collection stages. If you're not sure, you should investigate the matter further, for example by asking additional information to the people who collected the data (hoping that they won't be trying to cover their tracks if it's a case of fraud from their part).
As a side note, you can find various references to prevent or detect problems related to survey fraud or falsification (e.g. Schwanhäuser, S., Sakshaug, J. W., & Kosyakova, Y. (2022). How to catch a falsifier: Comparison of statistical detection methods for interviewer falsification. Public opinion quarterly, 86(1), 51-81.)