Skip to main content
deleted 35 characters in body
Source Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard deviation for each attribute, and I would like to order them by "variability" (roughly, that is - if the concept I'm driving at is not a statistical one, I'll settle for one that is just intuitively appealing.) Obviously I can rank them by sigma, highest to lowest (ranging from very high to very low in absolute terms, on the order of thousands down to <1.0). But is that meaningful? Does a large sigma (or variance) mean it's more "variable", or should I be looking for the largest RATIO of sigma to mean? I can't find any reference to a concept like that in my textbooks, but it seems to me to convey more meaning than a variance/sigma on its own. (The point, if you haven't guessed, is to reduce the dimensionality to something more manageable by discarding the attributes with the least variability.)

Thanks very much, and cheers - Ed

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard deviation for each attribute, and I would like to order them by "variability" (roughly, that is - if the concept I'm driving at is not a statistical one, I'll settle for one that is just intuitively appealing.) Obviously I can rank them by sigma, highest to lowest (ranging from very high to very low in absolute terms, on the order of thousands down to <1.0). But is that meaningful? Does a large sigma (or variance) mean it's more "variable", or should I be looking for the largest RATIO of sigma to mean? I can't find any reference to a concept like that in my textbooks, but it seems to me to convey more meaning than a variance/sigma on its own. (The point, if you haven't guessed, is to reduce the dimensionality to something more manageable by discarding the attributes with the least variability.)

Thanks very much, and cheers - Ed

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard deviation for each attribute, and I would like to order them by "variability" (roughly, that is - if the concept I'm driving at is not a statistical one, I'll settle for one that is just intuitively appealing.) Obviously I can rank them by sigma, highest to lowest (ranging from very high to very low in absolute terms, on the order of thousands down to <1.0). But is that meaningful? Does a large sigma (or variance) mean it's more "variable", or should I be looking for the largest RATIO of sigma to mean? I can't find any reference to a concept like that in my textbooks, but it seems to me to convey more meaning than a variance/sigma on its own. (The point, if you haven't guessed, is to reduce the dimensionality to something more manageable by discarding the attributes with the least variability.)

Source Link
EdB
  • 21
  • 2

Which distribution has the larger variability? Is variance absolute, or relative to the mean?

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard deviation for each attribute, and I would like to order them by "variability" (roughly, that is - if the concept I'm driving at is not a statistical one, I'll settle for one that is just intuitively appealing.) Obviously I can rank them by sigma, highest to lowest (ranging from very high to very low in absolute terms, on the order of thousands down to <1.0). But is that meaningful? Does a large sigma (or variance) mean it's more "variable", or should I be looking for the largest RATIO of sigma to mean? I can't find any reference to a concept like that in my textbooks, but it seems to me to convey more meaning than a variance/sigma on its own. (The point, if you haven't guessed, is to reduce the dimensionality to something more manageable by discarding the attributes with the least variability.)

Thanks very much, and cheers - Ed