From my perspective, the reason p-value of a statistical test isn't useful in large sample scenario is because it will change according to the scale.
E.g. let's focus on chi-square test. In a chi-square test, the chi-square score changes linearly w.r.t. the scale of the frequencies (I mean, multiply 10 for each cell in the contingency table, the chi-square will also be 10 times larger). So when the data size is larger, even it has the same ratio/proportion, the p-value becomes smaller.
Thus, in this sense, why don't we just set a standard value, e.g. 1000 or 10000, and do a linear transform to normalize the total sample size to this value before we do the chi-square test. Then I suppose the p-value makes sense again. And scores like Cramer's V or Cohen's h or odds ratio remains the same.
What's wrong with my idea? (I don't know why the number in the table must be counts for chi-square. If this is a problem, maybe think of other tests like ANOVA)
Any idea/thought is appreciated:)