0
$\begingroup$

I have a process 𝑃 generating random variables X_1, ... X_n. From each of these I've sampled a set of samples S_i = (s_i1, ... s_im), respectively.

I want to test whether the process/real-world dynamic generating the variables X_i is consistently (or pretty consistently) generating normally distributed variables (not identically distributed, though! They can have different μ and σ).

One suggestion I got was to think of all samples (S_1, ..., S_n) as coming from a gaussian mixture. However, it didn't come with a suggestion for a statistical test, and I couldn't find one.

My original thought was to test each sample separately for normality (with something like Shapiro-Wilks), control the False Discovery Rate in the set of tests (using something like Benjamini-Yekutieli; I can't assume independence), and then see whether the rate of rejections (in the case of Shapiro-Wilks, this means non-normality) is under/over some predetermined threshold (like 10% or 20%), in which case I deem the mysterious process as one not generating gaussians consistently enough.

However, I'm no statistician, so I've come to ask for help. Thank you! :)

Clarification:

One shouldn't think of X_i as if they were collected from the same process. 𝑃 is a process generating processes, if you want. Or distributions, or populations.

Think of 𝑃 as "my marketing team" and different X as different marketing campaings they output, where each such campaign generates different revenue streams, or engagement streams. One might be a tv campaign in Belfast, another an online campaign in Kenya. We thus do not expect the revenue/engaement/whatever distribution for all of them to have the same μ and σ. Therefore, the question is whether revenue streams from campaigns generally distribute normally or not.

Improve this question
$\endgroup$
6
  • $\begingroup$ It's not clear to me why, given that you've collected variables from a particular process and sampled from them apparently at random, you think the underlying distributions might be different. How did you generate your samples? Why would the underlying distributions for the samples be different, given that the population was generated by the (one) process $P$? $\endgroup$
    – jbowman
    Commented May 31, 2022 at 0:09
  • $\begingroup$ They were not collected from the same process. P is a process generating processes, if you want. Or distributions, or populations. Think of 𝑃 as "my marketing team" and different X as different marketing campaings they output, where each such campaign generates different revenue streams, or engagement streams. One might be a tv campaign in Belfast, another an online campaign in Kenya. Do you really expect the revenue/engaement/whatever distribution for all of them to have the same μ and σ? Thus, the questions is whether revenue streams from campaigns generally distribute normally or not. $\endgroup$
    – Shay Palachy Affek
    Commented May 31, 2022 at 9:31
  • $\begingroup$ Ah, that helps. You might want to rewrite your first couple of sentences to reflect that. $\endgroup$
    – jbowman
    Commented May 31, 2022 at 15:01
  • $\begingroup$ Right you are! Done and done! :) $\endgroup$
    – Shay Palachy Affek
    Commented Jun 2, 2022 at 9:01
  • $\begingroup$ How many random variables do you have, and how formal a solution do you need? You can of course inspect and separately test all individual samples, from which you can learn more than by just running a single overall test. Every adjustment for multiple testing has pros and cons, but how much do you need to bother? Formal tests are for specific decision problems, and I'm not sure if this is what you need. $\endgroup$
    – Christian Hennig
    Commented Jun 2, 2022 at 9:37

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.