You must always ask yourself, what quantities am I interested in? Statistics does not (directly) answer non-numerical questions. You must consider - which aspect of which group of people am I interested in, and how are they related to values in the sample at hand?
Descriptive statistics, such as the mean, the correlation coefficient, or Cohen's d, quantify various aspects of a sample.
Inferential statistics, such as point hypothesis tests, provide estimates of these very same measures for a whole population based based on a subset (in the face of sampling error). This allows one to guess what a descriptive statistic would have been, had the whole population been measured without error. They generalise from some data we have available, to some data we haven't - under the assumption that the data we have is representative of the data we haven't.
An inferential statistic will not be more than an approximation of a descriptive statistic. As noted by @january, Statistical significance does not mean practical significance; rather, statistical significance (in the context of point hypothesis tests) informs you that you may assign low confidence to a specific single value for the population parameter (often zero). If you precisely know the value of the population parameter, I cannot imagine any reason to estimate it.
What it means to you that you have low confidence in a given value for the population parameter, such as "rejecting the null" when p<0.05, typically meaning that you reject with some confidence the hypothesis that the population mean is 0, depends on the problem in question. The value may be non-zero, but so low as to have no practical relevance. That a test rejects a point null does not carry more direct information than knowing the population parameter to be a specific (non-zero) value; rather, it carries less information, since the hypothesis test sheds doubt on one value, the descriptive statistic sheds doubt on all but one values. (Indirectly, a p value will also inform you about other descriptive statistics, such as variance and effect size). You might imagine inferential statistics as "confidence labels" assigned to descriptive statistics (although this metaphor is getting dangerously close to p(H|D)).
However, it is not so easy as to say that if the whole population the researcher is interested in has been measured, descriptive statistics are unequivocally superior.
If inferential statistics make sense or not depends not only on the fraction of the population sampled, but also on the reliability of the measurements. For example, height in @January's example is rather easy to measure correctly (ignoring for now that people grow, die, have accidents, ...). But, what if you were interested in their memory span, income or beard length? In such cases, sampling error characterizes the data even though the population has been sampled, and you do not in fact precisely know the parameter. If you were to repeat the measurement, you would get completely different (though statistically very similar) results! In such cases, inference may still be useful.
But basically: consider which parameter you are interested in. p values and statistical significance aren't so much parameters, but "confidence labels" for such parameters.