2
$\begingroup$

I'm working with a dataset of elemental concentrations in polluted soils. Using the same units, some elements have high values and some have low values. If the concentration of some element is too low in any given soil it can't be determined, so I have missing values (they could be approximated, though), and polluted places have much higher values, so they are outliers and most variables are not normally distributed. I want to group soils with similar profiles, and if possible know what elements (variables) make them similar or different. The methods I know (like PCAs) can't be applied here, so what method could be?

$\endgroup$
1
1
$\begingroup$
  1. If you know for certain that your missing values only occur where an element is below the limit of detection (L.O.D.), I would argue that an approximation of half of the value of the L.O.D. would be better than treating them as missing values.

  2. Have you considered transforming your variables so you can use them in analyses that you're familiar with?

  3. Have a look at similarity matrices and hierarchical cluster analysis. If you already have defined groups that you know are similar, you may also find it worth taking a look at Random Forests - they can be used with categorical data as the response variable, and deal with various explanatory variable data types and missing values (although you still need to be careful to consider issues that these might cause in terms of bias, and take a look at conditional importance vs. the original variable importance score).

$\endgroup$
2
  • $\begingroup$ Both your answer and the article shared by proof_by_accident in the comments go along the same lines, I did not know that transforming variables is an acceptable way to do it, so I will try with that, thank you! $\endgroup$
    – Antón
    Sep 24 '20 at 7:41
  • $\begingroup$ Glad it helped. The article is good. I should also add that CART/decision tree methods (which underpin Random Forests) can be used for regression to predict a numerical value and so could also be used for grouping (you might need to restrict the number of 'leaves' on the tree so you don't end up with too many small groups). If I had lots of potential explanatory variables, I would consider running a Random Forest first to filter down to a smaller set. $\endgroup$
    – Izy
    Sep 24 '20 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.