I would start with a simpler model, for example logistic regression, in which you can look at the chi-square test to see how the addition of X
variables improves the model, as well as at the individual p-values. If you do not see statistically significant results especially for the entire model, this would suggest the variables are not very informative of y
. If you have reasons to believe there are interactions or higher-order terms (e.g. squares), you can add these to the model.
You could then progressively try more complex models (e.g. random forest or neural networks) and show that even with more flexibility (interactions etc.), the results do not improve. As per @Dave's helpful comment: to make the general statement that X
is not informative of y
, you could use a measure such as ROC-AUCROCAUC
, which does not depend on a specific threshold.
If itROCAUC
is very close to 0.5, it means that for all possible decision thresholds, it performs on averageyour classifier is not better than randomly flipping a coin flip. More precisely, if we let the classifier rank data points (highest ranking for most sure about positive class), the chance that a randomly chosen positive class will be ranked above a randomly chosen negative class is just 50%, which is no better than random order.
Finally, to make the case that X
is not strongly related to y
, it would help greatly to discuss what is in the data and why X
might not be informative at all. For example, if you are predicting lung cancer for patients and X
data contains job occupations, it could be the relationship is weak -- job occupation does not connect with lung cancer unless perhaps in very specific jobs like mining. However, if X
contains demographic data or information about smoking, it seems strange if it could not predict lung cancer to some extent.