I'm informally reviewing a study by a coworker, but as I'm not a specialist of survey weighting procedures, I need a second opinion about something I disagree with him.
He collected a convenience sample, from a (human) population on which we don't know much – except that this is a population we cannot treat as a finite population. He assigned survey weights to respondents, correcting on the variable "gender". He assumed that the distribution of men and women in the population of interest was (0.5, 0.5), while in the sample there are about 70% women. There are about 500 respondents.
However, his assumption of 50% men and 50% is based on a wild guess, as there's no existing data on this population of interest.
He wants to crosstabulate the variable gender against some others (e.g., variables with 2 or more categories), and then conduct chi-square tests of independence corrected for the weights he computed. He also wants to compute some other statistics based on these weights (e.g., the corrected average age).
His justification for this procedure is that he wants to compare the sample to a hypothetical scenario where there would be an equal number of men and women in the population. I suggested that maybe we wanted to test the hypothesis that the sample was coming from a population where there's a (0.5,0.5) distribution (using a goodness-of-fit test), but that's not what he wants.
He wants to present the results from this procedure to stakeholders who requested the study. He plans to present both the "corrected" and "non corrected" results. To me, as nothing really justifies a scenario of 50% men/50% women (for example, why not 60% vs 40% instead?), I feel that this would bring more confusion than information.
He insists that his procedure is valid, so now I'm wondering if he might have a point. So... does he? Is there any reference on such a procedure?
Here is some additional information, as requested in comments:
The population is comprised of tourists in the small towns of a quite touristy area in a European country (about 30 millions of tourists per year in this area). Unfortunately even available data about tourists in general in the country (or in this area) offer no information relative to gender (or other socio-economic characteristics for that matter, barring their country/continent of origin). Even if such data was available, I'd be wary about treating tourists in small towns of this area as being like tourists in general (they might even differ from tourists in larger cities in the same area!).
Following another remark in the comment section: I agree it's difficult or impossible to make inference from a convenience sample in the first place - so the question of using this weighting procedure with "imaginary" data may be secondary or irrelevant. However, my worry is that we could try this weighting procedure with many different hypothetical distributions, so why choosing specificallly 50/50%? I have serious doubts about how informative it is (I think there's a risk of involuntarily misleading the stakeholders, by having them thinking that the "corrected" data are more reliable than the "non corrected" data), but I'm also wondering if I'm being overcautious here or if I'm missing something.