Really need to know this from someone who really understands statistical significance. Can we apply Statistical Significance when reading the results of AI driven marketing?

Challenge: Large proportion of advertisement at this age is driven by AI. Hence, inputs are not randomly evenly picked, AI are constantly optimizing and picking the best users to show the advertisement based on its own understanding, and the advertisement is also displayed at many different locations in different orders depending on the learning stage of the AI.

With the AI now picking the inputs then(no longer random), When we try to read/understand the real performance of the Advertisement, can we still use the concept of Statistical Significance?

Additional Background, this blog post says it's not applicable anymore, and prove it with his experiment: Debunking Ad Testing Part 1: Statistical Significance

  • 1
    $\begingroup$ What is described as "best practice" in the first figure in the linked post is known to be poor practice and has been for a very long time. The author is kicking down an open door. $\endgroup$ – einar Aug 19 '20 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.