While I mostly have a good understanding for the mathematical aspects of statistics, I find myself struggling with seemingly arbitrary choices:
For example I have a 2D matrix of chemical compounds showing whether they co-locate or not. (binary 0/1)
To reduce the noise, I want to remove outcomes (0/1) that have a probability over 90% at random. (for example based on concentrations and random movement)
Now how can I ever justify why I decided for a 90% cut-off and not 75% or 99% instead? Do you have any resources that help with such "arbitrary" choices and how to defend against potential claims of data-manipulation? I wonder how you statistics veterans deal with such decisions!
(Basically whatever I do, it will look like I screened through every cut-off and selected whichever looks best for the scientific publication.)