New answers tagged experiment-design
0
votes
A/B testing and Experiment Length
a rule of thumb formula for minimum sample size is: $n = \frac{16 * \sigma^2}{\delta^2}$, there are two things that need more clarify:
n is the minimum sample size for each group: if you have an ...
- 311
0
votes
regarding generating synthetic data simulating the real data
If you had some real data, then you could leverage GMMs or Generative AI models to create synthetic data to mimic your data. If you don't have any real data, then could go for declarative approaches, ...
1
vote
Optimizing Treatment Group Assignments
if you hava few and categorical covariates variable(pre-experiment measurements) , you can try block: This approach achieves almost perfect balance between covariates, and degrees of freedom don't ...
- 311
0
votes
AB testing, which denominator to use for lower funnel metrics
This is a very good question. Let's first outline a few assumptions and draw a DAG.
First, it seems like users are randomized after arriving at the checkout page. This means the "treatment&...
- 31.3k
0
votes
What are several examples of survey questions which have response options designed as mutually exclusive when they are not?
One classic mistake is assuming that something is a binary, when it's not. Giving only two options, assumed to be mutually exclusive and complete, makes the survey difficult to answer when reality is ...
3
votes
What are several examples of survey questions which have response options designed as mutually exclusive when they are not?
A common example is discrete counts with overlapping class-limits. For example: How many days did you visit Stack Exchange in the last month?
0
1
2-5
5-10
more than 10
The problem is this...which ...
0
votes
What are several examples of survey questions which have response options designed as mutually exclusive when they are not?
This is a community editable wiki.
Feel free to insert additional examples at the bottom
Example One: Video Content
The Question
Which of the following are you?
Video Content Creator
Business ...
0
votes
regarding generating synthetic data simulating the real data
In a similar case, I have used the following approach. I had some ideas about the characteristics of the synthetic data I wanted to generate.
I started searching for these characteristics in public ...
- 301
0
votes
Calculating sample size when you have 2 treatments and uneven distribution
Main point: use a standard RCT methodology but adjust your $\alpha$ level accordingly to account for multiple comparisons.
More details: Irrespective of how many arms we have and their respective ...
- 41.1k
0
votes
Can I apply both winsorization and CUPED to my experiment results?
I think you can apply both.
First,apply winsorizing on pre-experiment data and post-experiment data.
Second,use the winsorized pre-experiment data and winsorized post-experiment data to calculate ...
1
vote
Accepted
Does this example violate the independence assumption?
Yes, it is very likely. The evaluations for each employee are likely to be correlated, and if the same supervisor writes multiple evaluations, those are likely correlated. Each evaluation is in a ...
- 505
1
vote
Does this example violate the independence assumption?
These data points are almost certainly not independent. It doesn't matter if there are many evaluations are written by the same person, as there are certainly many evaluations written about the same ...
- 7,196
1
vote
When should we include the "Replicate" in an ANOVA table for Factorial Design?
We are given that each factor is three levels, two replicates, and that the treatment sum of squares is 1200.
The interpretation of this old question might be a bit ambiguous. I'll take it to mean ...
- 81.8k
2
votes
Accepted
How do Fractional Factorial Designs relate to 'Response Surface Methodology'?
A brief summary of what I've learned:
Response Surface Methodology (RSM) is a type of adaptive or sequential design. Sequential designs involve multiple rounds of experimentation, with the choice of ...
- 15.9k
Top 50 recent answers are included