All Questions
Tagged with randomization or random-allocation
211 questions
0
votes
0
answers
5
views
How should I balance an experiment with two sets of conditions and not enough trials for every combination of those conditions?
I have a within-subject experiment that tests participants' memory. There are 80 trials, and in each trial, participants read 1 of 20 sentences. Then, they have to manipulate the sentence in 1 of 4 ...
- 1
3
votes
1
answer
37
views
RWE: use of target trial emulation framework: why aiming to emulate RCT?
Hernán and Robins [1] introduced the target trial emulation framework in 2016 to define the question of interest in observational studies. The target trial framework asks the investigator to specify ...
- 337
1
vote
1
answer
57
views
Randomization and Causal Dags
Suppose we have Treatment T and Outcome O, and variables A and B, with the following causal dag:
But if we were doing an RCT with these treatment and outcome variables, is it accurate to think of the ...
- 125
2
votes
1
answer
41
views
Randomization validity based on a modulus function
I want to run an AB test on a platform that only supports assigning variants based on address_id. Each user can have multiple address_ids in our platform i.e each user_id can be mapped to multiple ...
- 21
1
vote
0
answers
26
views
Is the permutation test possible even when the randomization scheme is a restricted randomization?
Imagine an experimental setting where the participants are assigned randomly to some treatment groups. However, a form of restricted randomization is used (known by the experimenter).
After the ...
- 51
4
votes
1
answer
76
views
Experiment design to determine effect of ads on etsy shop performance
I am looking to design an experiment to determine the impact of turning on etsy ads for my shop. I opened a shop for plant pots and have made 27 sales in the 4 months I have been open. I am ...
- 41
2
votes
1
answer
41
views
Longitudinal analysis of peer effect
I am working with hierarchical data (2 repeated observations on children nested within households), obtained from a RCT with 2 treatment groups. The primary goal of my analysis is to see whether the ...
- 33
1
vote
1
answer
45
views
a few questions on stratified analysis
If one of the 3 stratification factor from IWRS has data errors, should we replace it with CRF collected baseline values? Will this break the balance of the strata blocks?
For non ITT population, say,...
- 529
1
vote
1
answer
30
views
Power estimate for an interaction term in factorial design
I am trying to estimate power for a factorial design (2x2).
My initial approach was to estimate pair-wise minimum detectable effect sizes : i.e., if I know sample size, allocation to each group and ...
- 317
7
votes
3
answers
1k
views
Why should you want to use a smaller control group?
In an RCT with multiple treatment arms, I heard that the "pure" control group be small to if we expect the effect size to be large. Why would this be the case? I understand that we could ...
- 317
4
votes
2
answers
146
views
Analyzing an experiment which consists of many "small" experiments
Consider the following setting: I am trying to choose between two bakers for my new bakery, Baker A and Baker B. The way that I am evaluating them is on a set of different cakes on the menu: Cake 1, ...
4
votes
1
answer
160
views
When and how of stratification?
I understand stratification at a novice level. For example, if we want to condition on gender in post experimentation inference, we might stratify or block on gender. As I understand it, we take each ...
- 3,520
2
votes
0
answers
78
views
Random permuted block sizes in a randomised clinical trial
What is the best way to choose the block sizes in a randomised clinical trial?
Randomisation for an ongoing trial is stratified by site (2 sites) and gender (m/f) for 4 strata - it uses random ...
- 185
4
votes
3
answers
154
views
Sample size calculation for a multicenter RCT
I'm very new to the world of statistics, so sorry in advance for my limited knowledge. I hope you don't mind helping me.
We are currently setting up a multicenter RCT of treatment A versus placebo in ...
1
vote
1
answer
51
views
High power in A/A test, interpretation?
Randomization has already been executed and two groups have already been assigned, where treatment observations and control observations are captured by df1, df2 respectively.
Note: Treatment exposure ...
- 3,520
1
vote
1
answer
35
views
Show conditional independence in randomized experiment
I'm new to causal inference, and I have a question regarding a potential causal relationship. I'm wondering if the following assumption holds:
$(Y_i(1), Y_i(0), X_i) ⊥ Z_i -> (Y_i(1), Y_i(0)) ⊥ Z_i ...
- 11
0
votes
2
answers
129
views
What's the magic of randomization? [closed]
I've read that one can perform A/B tests & effect size inference given that a population was randomly sampled and allocated to A & B groups. You simply give the treatment to group B and the ...
- 3,520
18
votes
3
answers
979
views
Causal diagrams necessary in randomized controlled trials?
I understand how a properly administered RCT rules out confounders because there are no variables influencing the treatment/control group assignment except randomization (meaning no backdoor paths ...
- 6,286
2
votes
1
answer
70
views
Estimating Complier Average Causal Effect (CACE) in a Field Experiment with One-Sided Noncompliance
I am working on a referee report for a field experiment focused on the employment process. The experiment investigates if including a video in job applications can reduce employment disadvantages for ...
1
vote
1
answer
154
views
Can we perform a MANCOVA for paired data?
I have data for a cross-over randomized controlled trial. I wonder if I can use MANCOVA to compare my outcomes (they are continuous) while having paired data. If I can't, what would be an appropriate ...
- 33
5
votes
1
answer
177
views
How should I analyse this experiment with two blocking factors
In this experiment, a machine is used to compare the wear resistance of 4 different types of rubber-coated fabric (variable mat coded A, B, C and D). To do this, ...
- 323
3
votes
1
answer
393
views
What is a Randomized Complete Block Design. How do we create one and analyze it?
I have been reading about experimental design, and Randomized Complete Block designs (RCBD) in particular and I have a few queries. What exactly is a randomized complete block design ? Is there a ...
- 323
0
votes
1
answer
97
views
Design of experiment in multiple regression
Suppose we have the following model of our environment:
$\hat{y}_t = e^{dayofweekeffect} * x_{1, t}^{\beta_0} * x_{2, t}^{\beta_1}$
which we can linearize into: $log(\hat{y}_t)= dayofweekeffect + \...
- 3
2
votes
1
answer
194
views
Sample size calculation for a three independent arm RCT with a categorical outcome
I have three study arms on interventions A, B, and C, which are independent of each other. The primary outcome is categorical (positive or negative). The null hypothesis is A = B = C.
From prior ...
- 340
6
votes
1
answer
182
views
No more than $n$ moose, but how many?
Introduction
I am thinking about how to estimate the number of individual moose from wildlife camera photos. I have the latitude and longitude position of each observation, along with a datetime of ...
- 9,660
3
votes
2
answers
87
views
Time clusters in a randomised trial
In cluster randomised trials, some clusters are treated whereas others are not. Usually, the clusters are geographical unit, e.g. villages. However, am I right that, in principle, the clusters could ...
- 307
6
votes
1
answer
70
views
Ran out of envelopes in randomized control trial
We are conducting a randomized control trial (RCT) comparing treatment vs. control from 4 sites (A-D). We have planned for 100 patients from each site. 100 envelopes of treatment assignment (...
- 923
1
vote
0
answers
39
views
Do I understand correctly how a split system should work in AB tests
I have a question about how a split system should work.
I have a product. Which looks like eBay.
We conducted an experiment that should increase the number of users who choose one of the categories.
...
1
vote
1
answer
138
views
Randomization of questions in an online survey
I want to run an online survey about people's preferences for environmental policies in different economic sectors using a Likert-type scale (strongly disagree, disagree, etc.). I have 4 sectors and 4 ...
- 55
1
vote
1
answer
68
views
Which is the correct way to deal with insignificant main effect of Group condition? Stick with interaction effect in RM-Anova or perform ANCOVA?
I wanted to ask conceptual about what to do with main effects?
Assume, I have randomly assigned equal two groups(let's say CBT, Control; N1=25; N2:25). I collected the depression levels at three time ...
- 11
3
votes
1
answer
47
views
Optimizing Treatment Group Assignments
When setting out experiments, I want to make sure treatment groups are as balanced as possible. Instead of using randomization, I've started to use the following process. I first collect some ...
- 297
3
votes
1
answer
391
views
Why is covariate balance fundamental for causal inference in observational data but unnecessary in experimental data?
The observational case
For observational data, Hernán & Robins (2023, p. 49) state:
In the absence of marginal randomization, [computing the average causal effect in the entire population] ...
- 1,592
1
vote
0
answers
22
views
Causal identification when some ITEs are 0
Say I have some data from an RCT related to a skills training program. The data are such that:
Marginally randomized binary treatment z. Those with ...
- 47
7
votes
4
answers
1k
views
Why is the Central Limit applicable in A/B testing?
I am having trouble understanding why the Central Limit Theorem (CLT) is applicable in A/B testing. As a beginner in statistics, I am trying to grasp the intuition behind it.
The CLT states that as we ...
- 71
1
vote
1
answer
132
views
Understanding the advantages of CRD experiments
I am self studying an introductory course on Designed Experiments and have come across the notion of a Completely Randomised Design (CRD) defined as follows:
Completely Randomised Design: the ...
- 243
0
votes
1
answer
179
views
How much of a discrepency is tolerable if Sample Ratio Mismatch is detected?
We have analyzed several past online A/B tests and revealed that some of them have SRM at a statistically significant level (p<0.001).
Example: we had 50%/50% traffic allocation between control and ...
- 13
0
votes
0
answers
30
views
What’s a good scale robust similarity metric?
Context: While designing an experiment, there is interest from leadership in pairing control and test units together such that every (test, control) pair are as categorically similar as possible.
...
- 3,520
1
vote
0
answers
17
views
Why training is smoother when no randomization of batch after each epochs is applied?
I am wondering why training error converges more smoothly when I am not applying randomization after each epochs?
After each epoch I am taking the loss obtained in my last minibatch. I see that when ...
- 11
1
vote
1
answer
281
views
Equal size random assignment
I am doing a ab test. I want to test if a treatment is statistically better than a control. I want to split 30 subjects randomly and evenly into two groups. My way is first generate random number for ...
- 11
0
votes
1
answer
74
views
Balanced incomplete block design - randomizing treatment order
In a balanced incomplete block (BIB) design, what is the best way to ensure each treatment appears in all positions/orders? Should I randomize treatment order within each block, so that participants ...
- 1
3
votes
1
answer
119
views
Medical experimental design with non-comparable control
My population consists of people at high risk of cardiovascular disease who are identified through standard blood tests. These people, when identified, are prescribed some standard medication.
I will ...
- 7,958
0
votes
0
answers
71
views
Help Needed for Outliers detection post paired T-test statistical test
I don't know if this is a standard way od doing things so open to any suggestions, basically I have done random sampling from my population to create 2 groups Treatment & Control. I also have few ...
- 143
0
votes
0
answers
32
views
What statistical design and analysis is appropriate for this single-arm study?
I have research to perform but I am not very familiar with statistical design and the relevant analysis that I have to do, but here is the idea of what I want to do:
It is a single-arm study, I have a ...
- 1
5
votes
2
answers
307
views
Switching interventions after randomization
Suppose in a randomized control study, we assigned each individual to one of two interventions (A or B) with equal probability. And suppose for the sake of argument, there is complete balance in ...
- 641
1
vote
0
answers
75
views
Test statistics for randomization
I have a basic question on what method should be applied to obtain test statistics distribution.
I have two treatments $A$ and $B$. I want to test on mean $H_0: \overline{t}_A=\overline{t}_B$ as ...
- 1,465
2
votes
1
answer
97
views
Bootstrapping confidence intervals in randomization model and population model
Ernst 2004 shows that the permutation tests in the random assignment scheme (e.g., controlled experiment) and in the random sampling scheme (e.g., observational study) share the same constructing ...
- 381
2
votes
1
answer
676
views
In a randomized trial, should we exclude random intercepts and use only slopes?
Let's say I have a longitudinal study, with patients assessed at several time points, which goal is to compare the treatment vs. placebo.
If, theoretically, I used a mixed model to analyse the ...
- 41
1
vote
1
answer
45
views
AB Test, Experimental Design based on Machine Learning Output
Looking for the best way to set up an AB test.
Scenario is that the online product I am working with has logged in users that pay a monthly fee, and they can upgrade to get access to more features.
...
- 11
0
votes
1
answer
62
views
Balance checking (using discrete variables)
Context
I'm doing a field intervention, with 4 treatments and 1 control. I have done a stratified randomization beforehand to ensure that demographic characteristics (gender, age, income group) are ...
- 55
2
votes
1
answer
120
views
RCT using simple Randomisation, can extra participants be added?
I have been asked to perform the randomisation for a small RCT using simple randomisation. There are 100 individuals to randomise to one of two arms (control vs intervention). The researchers have not ...
- 333