I am an MBA student taking some courses in statistics.
We attended a seminar on GLM Models for Count Data in which the presenter was introducing us to the concept of "Nuisance" Parameters.
I am not sure that many of us fully understood the idea - but what I got out of it, was the following: If you have some Normally distributed data and you want to estimate the "mean" of this data, you have to indirectly estimate the "variance" at the same time - therefore, in this case, the "variance" can be considered as a "nuisance" parameter.
Assuming I understood all this correctly, I am left with this question:
Why exactly is the "variance" considered as a "nuisance"? Is it because we now have to do more work to estimate the "mean"?
And using this logic, aren't almost all parameters a "nuisance" in some way? For example, take the same question - but this time, suppose I want to estimate the "variance". Could I now consider the "mean" as a "nuisance" parameter"?
I feel like I am not fully/correctly understanding the idea behind "nuisance" parameters - these alleged "nuisance" parameters don't really seem like that much of a "nuisance" to me. But doing a Google Search reveals so much niche statistical research that has been done on estimation and "nuisance" parameters - thus I feel I am wrong about this, and that somehow "nuisance" parameters can be far more of a "nuisance" than I had initially anticipated.
PS: I hope I haven't caused too much of a "nuisance" when posting this question!