Aren't ALL Parameters Eventually "Nuisance Parameters"?

Question

I am an MBA student taking some courses in statistics.

We attended a seminar on GLM Models for Count Data in which the presenter was introducing us to the concept of "Nuisance" Parameters.

I am not sure that many of us fully understood the idea - but what I got out of it, was the following: If you have some Normally distributed data and you want to estimate the "mean" of this data, you have to indirectly estimate the "variance" at the same time - therefore, in this case, the "variance" can be considered as a "nuisance" parameter.

Assuming I understood all this correctly, I am left with this question:

• Why exactly is the "variance" considered as a "nuisance"? Is it because we now have to do more work to estimate the "mean"?

• And using this logic, aren't almost all parameters a "nuisance" in some way? For example, take the same question - but this time, suppose I want to estimate the "variance". Could I now consider the "mean" as a "nuisance" parameter"?

I feel like I am not fully/correctly understanding the idea behind "nuisance" parameters - these alleged "nuisance" parameters don't really seem like that much of a "nuisance" to me. But doing a Google Search reveals so much niche statistical research that has been done on estimation and "nuisance" parameters - thus I feel I am wrong about this, and that somehow "nuisance" parameters can be far more of a "nuisance" than I had initially anticipated.

PS: I hope I haven't caused too much of a "nuisance" when posting this question!

Answer 1

The concept of nuisance parameters is somewhat controversial, and some argue it should not be used (I do not understand that argument). It might be better to focus on the opposite concept of interest parameters (also called focus parameters, see for instance Changing the size of a confidence interval in order to emphasize results)

Start with what is your research question. If that can be formulated as a question about one (or a few parameters), those are the interest parameter(s). If it can only be formulated as a question about some complicated function of the parameters, consider a reparametrization so your focus/interest parameter is actually a parameter in the model.

Now, for the nuisance parameters, they are just those left after you have selected your interest parameter(s). There are many posts on this site about nuisance parameters, so search! Some:

• When do we have nuisance parameters?

• Is there a radical difference in how bayesian and frequentist approaches treat nuisance parameters?

For your normal means example, if your research question is only about the mean, then the variance parameter $\sigma^2$ is a nuisance parameter. It does not influence your hypothesis/research question directly, but it determines the precision with which you estimate the mean. But be careful, if the variance is not a constant (for instance, varies between different groups in the data), then variance might well be of interest after all. See Unequal variance in randomized experiments to compare treatment with control? and references therein.