Skip to main content
added 149 characters in body
Source Link
Nick Cox
  • 59.5k
  • 8
  • 136
  • 212

It's a good question, because in practice a great deal depends on experience rather than exact rules, and how is one to judge with little or no experience, and who counts as experienced or expert, and will experts always agree? (They won't.)

Even more depends on knowing that equal variances are an ideal condition, not a binding essential such that tiny deviations are fatal. It's a hobby-horse of mine, although not an original point, that the almost universal use of the term assumption in these statistical contexts is not especially helpful. In logic and pure mathematics, a failure of assumptions can be utterly fatal to the validity of an argument. In applied mathematics, including statistics, a failure of "assumptions" has to be judged pragmatically, because just about every application is an approximation. We would often be better off talking about ideal conditions, a phrase intended to march with a realisation that real data are usually messy and imperfect, especially when compared with fantasy or brand-name distributions. (Ironically, or otherwise, one of the most important ideal conditions, independence in some sense, is rarely discussed or checked for.)

I think thereThere are some slightly more precise guidelines that I would add.

  1. As a starting point, unequal variances that need attention tend to leap out at you from a plot, sometimes phrased in terms of a pattern hitting you between the eyes. If you're in doubt, you can usually assume there isn't a real problem.

  2. Perhaps contradicting #1, but that's typical of any advice: you can't always trust a graph. An appearance of greater or lesser variability can sometimes arise from differences in group size. A large group is more likely to include values from the tails of a conditional distribution than a small group. Hence if in doubt, calculate the variances to check, either for pre-defined groups as here or in some other appropriate manner.

  3. Is there a better model within reach? is the important associated question. For example, if variability of residuals seemed to increase with fitted or predicted values, I might wonder about working on a transformed scale, say by taking logarithms or (even better) using a generalized linear model with a logarithmic link. You stick with a model if you can't think of a better one, or more positively, change to a better model if you can see one. (Trying another model and finding that it isn't better, indeed possibly worse, and being able to report that, is very good practice in my view. Some people get queasy about choosing a model after exploration of the data or initial analysis. The view that you must think up a model in advance rather limits the scope for learning from data. Where did the model come from any way?)

  4. Your data are unlikely to be absolutely unique or unprecedented. What do people do in your journal literature? More generally, what do you know, as a scientist or other subject-matter expert, about how big or how small values may be, including whether there are limits to counted or measured values?

  5. More work, but not much so with decent software, is to simulate with similar sample sizes with datafrom a set-up with homoscedastic errors and see how different do results look in a portfolio of fake datasets? People new to statistics often underestimate how much variability there is in small samples, even if the underlying process is close to ideal. This example qualifies as a very small dataset by most standards.

It's a good question, because in practice a great deal depends on experience rather than exact rules, and how is one to judge with little or no experience, and who counts as experienced or expert, and will experts always agree? (They won't.)

Even more depends on knowing that equal variances are an ideal condition, not a binding essential such that tiny deviations are fatal. It's a hobby-horse of mine, although not an original point, that the almost universal use of the term assumption in these statistical contexts is not especially helpful. In logic and pure mathematics, a failure of assumptions can be utterly fatal to the validity of an argument. In applied mathematics, including statistics, a failure of "assumptions" has to be judged pragmatically, because just about every application is an approximation. We would often be better off talking about ideal conditions, a phrase intended to march with a realisation that real data are usually messy and imperfect compared with fantasy or brand-name distributions.

I think there are some slightly more precise guidelines that I would add.

  1. As a starting point, unequal variances that need attention tend to leap out at you from a plot, sometimes phrased in terms of a pattern hitting you between the eyes. If you're in doubt, you can usually assume there isn't a real problem.

  2. Perhaps contradicting #1, but that's typical of any advice: you can't always trust a graph. An appearance of greater or lesser variability can sometimes arise from differences in group size. A large group is more likely to include values from the tails of a conditional distribution than a small group. Hence if in doubt, calculate the variances to check, either for pre-defined groups as here or in some other appropriate manner.

  3. Is there a better model within reach? is the important associated question. For example, if variability of residuals seemed to increase with fitted or predicted values, I might wonder about working on a transformed scale, say by taking logarithms or (even better) using a generalized linear model with a logarithmic link. You stick with a model if you can't think of a better one, or more positively, change to a better model if you can see one. (Trying another model and finding that it isn't better, indeed possibly worse, and being able to report that, is very good practice in my view. Some people get queasy about choosing a model after exploration of the data or initial analysis. The view that you must think up a model in advance rather limits the scope for learning from data. Where did the model come from any way?)

  4. Your data are unlikely to be absolutely unique or unprecedented. What do people do in your journal literature? More generally, what do you know, as a scientist or other subject-matter expert, about how big or how small values may be, including whether there are limits to counted or measured values?

  5. More work, but not much so with decent software, is to simulate with similar sample sizes with data with homoscedastic errors and see how different do results look in a portfolio of fake datasets? People new to statistics often underestimate how much variability there is in small samples, even if the underlying process is close to ideal. This example qualifies as a very small dataset by most standards.

It's a good question, because in practice a great deal depends on experience rather than exact rules, and how is one to judge with little or no experience, and who counts as experienced or expert, and will experts always agree? (They won't.)

Even more depends on knowing that equal variances are an ideal condition, not a binding essential such that tiny deviations are fatal. It's a hobby-horse of mine, although not an original point, that the almost universal use of the term assumption in these statistical contexts is not especially helpful. In logic and pure mathematics, a failure of assumptions can be utterly fatal to the validity of an argument. In applied mathematics, including statistics, a failure of "assumptions" has to be judged pragmatically, because just about every application is an approximation. We would often be better off talking about ideal conditions, a phrase intended to march with a realisation that real data are usually messy and imperfect, especially when compared with fantasy or brand-name distributions. (Ironically, or otherwise, one of the most important ideal conditions, independence in some sense, is rarely discussed or checked for.)

There are some slightly more precise guidelines that I would add.

  1. As a starting point, unequal variances that need attention tend to leap out at you from a plot, sometimes phrased in terms of a pattern hitting you between the eyes. If you're in doubt, you can usually assume there isn't a real problem.

  2. Perhaps contradicting #1, but that's typical of any advice: you can't always trust a graph. An appearance of greater or lesser variability can sometimes arise from differences in group size. A large group is more likely to include values from the tails of a conditional distribution than a small group. Hence if in doubt, calculate the variances to check, either for pre-defined groups as here or in some other appropriate manner.

  3. Is there a better model within reach? is the important associated question. For example, if variability of residuals seemed to increase with fitted or predicted values, I might wonder about working on a transformed scale, say by taking logarithms or (even better) using a generalized linear model with a logarithmic link. You stick with a model if you can't think of a better one, or more positively, change to a better model if you can see one. (Trying another model and finding that it isn't better, indeed possibly worse, and being able to report that, is very good practice in my view. Some people get queasy about choosing a model after exploration of the data or initial analysis. The view that you must think up a model in advance rather limits the scope for learning from data. Where did the model come from any way?)

  4. Your data are unlikely to be absolutely unique or unprecedented. What do people do in your journal literature? More generally, what do you know, as a scientist or other subject-matter expert, about how big or how small values may be, including whether there are limits to counted or measured values?

  5. More work, but not much so with decent software, is to simulate with similar sample sizes from a set-up with homoscedastic errors and see how different do results look in a portfolio of fake datasets? People new to statistics often underestimate how much variability there is in small samples, even if the underlying process is close to ideal. This example qualifies as a very small dataset by most standards.

Source Link
Nick Cox
  • 59.5k
  • 8
  • 136
  • 212

It's a good question, because in practice a great deal depends on experience rather than exact rules, and how is one to judge with little or no experience, and who counts as experienced or expert, and will experts always agree? (They won't.)

Even more depends on knowing that equal variances are an ideal condition, not a binding essential such that tiny deviations are fatal. It's a hobby-horse of mine, although not an original point, that the almost universal use of the term assumption in these statistical contexts is not especially helpful. In logic and pure mathematics, a failure of assumptions can be utterly fatal to the validity of an argument. In applied mathematics, including statistics, a failure of "assumptions" has to be judged pragmatically, because just about every application is an approximation. We would often be better off talking about ideal conditions, a phrase intended to march with a realisation that real data are usually messy and imperfect compared with fantasy or brand-name distributions.

I think there are some slightly more precise guidelines that I would add.

  1. As a starting point, unequal variances that need attention tend to leap out at you from a plot, sometimes phrased in terms of a pattern hitting you between the eyes. If you're in doubt, you can usually assume there isn't a real problem.

  2. Perhaps contradicting #1, but that's typical of any advice: you can't always trust a graph. An appearance of greater or lesser variability can sometimes arise from differences in group size. A large group is more likely to include values from the tails of a conditional distribution than a small group. Hence if in doubt, calculate the variances to check, either for pre-defined groups as here or in some other appropriate manner.

  3. Is there a better model within reach? is the important associated question. For example, if variability of residuals seemed to increase with fitted or predicted values, I might wonder about working on a transformed scale, say by taking logarithms or (even better) using a generalized linear model with a logarithmic link. You stick with a model if you can't think of a better one, or more positively, change to a better model if you can see one. (Trying another model and finding that it isn't better, indeed possibly worse, and being able to report that, is very good practice in my view. Some people get queasy about choosing a model after exploration of the data or initial analysis. The view that you must think up a model in advance rather limits the scope for learning from data. Where did the model come from any way?)

  4. Your data are unlikely to be absolutely unique or unprecedented. What do people do in your journal literature? More generally, what do you know, as a scientist or other subject-matter expert, about how big or how small values may be, including whether there are limits to counted or measured values?

  5. More work, but not much so with decent software, is to simulate with similar sample sizes with data with homoscedastic errors and see how different do results look in a portfolio of fake datasets? People new to statistics often underestimate how much variability there is in small samples, even if the underlying process is close to ideal. This example qualifies as a very small dataset by most standards.