Questions tagged [experiment-design]
The study of how to structure an information-gathering exercise where variation is present.
1,807
questions
0
votes
0
answers
8
views
Stat experiment question on when to sample control/treatment
I was asked this question in a recent interview but basically the interviewer asked about when to sample control/treatment for an experiment. My apologies in advance as my memory on the question may ...
0
votes
1
answer
22
views
Statistical test for correctness of expectation
First time posting here.
Suppose I have a random variable $X$ and after various algebraic rewriting and calculus I have arrived at a closed form of $\phi = \mathbb{E}[X]$. However, I am not totally ...
- 3
0
votes
0
answers
25
views
Help Needed for Outliers detection post paired T-test statistical test
I don't know if this is a standard way od doing things so open to any suggestions, basically I have done random sampling from my population to create 2 groups Treatment & Control. I also have few ...
- 123
0
votes
0
answers
14
views
building design matrix of random effects in a LMM
I am working in simulations of data following a LMM. The general formula of the model, as defined in most texts books is as follows:
$$Y=X\beta+Zb+\epsilon$$
Where $b\sim N(0,D)$, $\epsilon \sim N(0,R)...
2
votes
1
answer
29
views
The correct treatment of censorship and observation periods in survival analysis using longitudinal observational data
I have longitudinal retrospective data and want to perform Cox proportional hazards regression. Still, I would be extremely grateful if the CrossValidated community could sanity-check my understanding ...
- 533
1
vote
2
answers
30
views
Identifying the best n points from a dataset
The problem I am trying solve involves identifying the best n data points from a dataset. For example if I have dataset with x1,x2,x3 independent variables and y1,y2 dependent variables I want to find ...
- 121
0
votes
0
answers
31
views
Cluster standard errors on experiment group?
I am running a very simple experiment. There are two different experiment groups, participants are randomly assigned. They have to make a simple binary decision in the experiment.
I have three control ...
- 243
0
votes
0
answers
5
views
Sample size for direct estimation of position bias in LTR
Imagine an AB test where control is my current recommender, which recommends 10 items. Treatment is a random shuffle of those 10 items.
My goal is to estimate position bias for each position while ...
- 83
0
votes
0
answers
21
views
Does Fisher exact test impose restriction on sampling design? [duplicate]
It is known that Fisher exact test assumes that all 4 margins of 2 by 2 table are fixed. $\chi^2$ test does not impose restriction on sampling design beforehand in general where I mean $\chi^2\sim\sum ...
- 1,179
2
votes
1
answer
14
views
Difference of baseline data between groups NOT significant on pre-matching [duplicate]
In a retrospective observational study, I'd like to compare the efficacy of drug A vs drug B, and was considering propensity score matching on age, gender, history of diabetes mellitus, and history of ...
- 23
0
votes
1
answer
24
views
A/B Testing Question - Power Analysis for Two Proportion Z Test
I'm having a hard time wrapping my head around experiment design and what appears to be a disconnect to experiment outcome analysis.
If I am proposing a z test of two independent proportions (...
- 3
0
votes
0
answers
14
views
Labeled orthogonal design in R
I have a problem preparing labelled orthogonal designs in the R program.
I have two labelled alternatives;
Four attributes are common for both of them but levels are different
One attribute is just ...
2
votes
3
answers
159
views
What does the phrase "adjusted by" mean?
"Matching of one group with specific exposures or outcome with a
comparable group is frequently used in medical research to investigate
the association between exposure and outcome." Ref
...
- 1,759
3
votes
0
answers
140
views
+100
What kind of study design is this and what kind of statistical analysis should be suitable?
I plan to conduct a study in which respondents will be asked to solve total 6 puzzles together.
Is there a specific study design (name) for this?
The scenario would be:
First, ...
1
vote
1
answer
26
views
Question on estimating (OLS) the ATE of RCT with multiple (2) treatments
Updated:
I do not have enough points to comment so...
Thank you Ben, you did interpret my question correctly. There are three treatment categories: control, A, and B. Thank you for clarifying that ...
- 11
1
vote
1
answer
23
views
How to do pairwise comparisons for Kruskal Wallis test?
If the null hypothesis in the Kruskal Wallis test is rejected, How can we perform pairwise comparisons? Can we apply Dunnett's Test for that?
- 11
1
vote
0
answers
25
views
Between Subjects Likert Analysis -- Treat as Ordinal or Continuous for Stats Test?
I have 4 groups in a between subjects design answering likert scale questions. I am wondering how to analyse the data. There are 20 questions for each user in each group of 30 (so 120 people all ...
0
votes
1
answer
20
views
Does this study design lend itself to a logistic regression?
I have a cohort comprising of individuals who participate in a weight-loss program and their weight measurements are collected longitudinally for 12 months (not necessarily at the same time points). ...
- 671
1
vote
1
answer
32
views
Regression Analysis of Time as Independent Variable in Experiment
we conducted an experiment on a sample of 30 persons. For every person, we observed the outcome variable every 15 minutes, so for each person 10 times.
IV: Time (in Minutes)
DV: Interval Scale [1;9]
n ...
- 13
2
votes
1
answer
63
views
Infer individual power contribution from team efforts
I like to infer the contribution of each rower in a crew boat from a
number of races: 8 rowers are split repeatedly into two boats of 4
rowers each. The race over a distance leads to an estimate of ...
- 223
5
votes
2
answers
51
views
How to choose number of sites and number sub-sites within each site in planning stage of a study? (ecology)
I have a situation in which I am going to perform a study on live blue mussels. The research question is how the prevalence of mussels affects the prevalence of other animals living on the sea floor.
...
- 53
0
votes
0
answers
15
views
What should I report on a 3-way ANOVA with a significant 3-way interaction effect?
I'm using graphpad prism to analyze some research data. The study design has 3 factors (Sex x Cardiac Anomaly x Exercise workload) for a variety of parameters. I don't use 3way ANOVA very often (and ...
- 1
0
votes
0
answers
14
views
When to use standard deviation versus standard error in linear error propagation
I have a question about linear error propagation. Let's say that I want to use an equation to calculate n, where n = (PV)/(RT) (eq.1)
I only take one measurement of P, and one measurement of T, but I ...
- 1
0
votes
0
answers
18
views
chi-squared for experimentation analysis - how to adjust for prior difference?
If using chi-squared test of independence to check if treatment group has affected the distribution of a binary outcome (e.g. click vs non-click), but there is also prior difference in click rate ...
- 171
2
votes
0
answers
18
views
Flip test and control group in causal inference to get sufficient sample sizes for synthetic control
There was supposed to be a buy-one-get-one (BOGO) promotion for sneakers at 5,000 stores. Unfortunately, a glitch happened, and 100 of the stores did NOT get the memo about the BOGO promotion, so they ...
- 61
0
votes
0
answers
10
views
Design for retrospective standardized testing and employment outcome study
we (student group) have been given a project involving about 10 years of corporate employment data, including variables on events like hiring date, training outcomes, promotion, retirement, etc. Also ...
4
votes
1
answer
63
views
Does it ever make sense to use the sum of the pre and post measure means?
Say I quantitatively measure a specific trait of each of the students in a group so that each of the students receive a score. The average score of the group is $X_0$. After a treatment, I measure a ...
- 91
0
votes
0
answers
30
views
Converting uncertain labels to probabilities for training data
I am devising a data collection effort to measure audio perceptual discrimination, and then model it using machine learning. What I am trying to model is: "Can a typical human hear the difference ...
- 101
2
votes
2
answers
45
views
Book recommendations for Combinatorial Designs
I am well-acquainted to basic topics of experimental designs such as anova, ancova, crd, rbd, lsd, factorial design, robust design etc (as per the book of Dr. D. C. Montgomery). Now as per my recent ...
2
votes
1
answer
47
views
Analysis of a campaign holdout experiment
We have an experiment where we are splitting an audience into 4 equal groups of 2000 customers each. 3 will receive an email but 1 group won't received anything. The goal is to test if they will ...
- 23
3
votes
2
answers
32
views
Determine if treatment effect is distributed equally across participants in a study or driven by a few participants
This is perhaps a malformed question, but I'll give it a shot. Let's say I have a pre and post measure of some outcome variable and a dummy variable treatment that ...
- 880
1
vote
2
answers
56
views
Correcting for pre-experiment bias in proportions test?
Say I have an obstacle course, which not everyone completes, though, globally, most do. I hypothesize that the treatment, drinking Gatorade, will cause an increase in the obstacle course completion ...
- 1,728
0
votes
1
answer
21
views
Notations in the statistical analysis of experimental research
I am very new to experimental setup and have recently started to learn about designing experiments and analyzing the results. In the papers that I read there are notations that I am unfamiliar with. I ...
- 221
2
votes
0
answers
22
views
How to study the impact of time in a mixed-factorial design?
I ran a study to compare 2 different computer interfaces (say A and B) meant to be used in an unfamiliar environment.
I want to compare:
How much effort participants took to complete a certain task ...
1
vote
0
answers
48
views
What layout is this?
A fleet of electric vehicles are to be investigated for the mileage performance per
charge. Four vehicles have been chosen randomly from the fleet and four observations of miles were made for each ...
- 11
0
votes
0
answers
32
views
Is "triple differences" and estimating "heterogeneous treatment effects" the same?
I am running a difference-in-differences (DiD) regression with a time and treatment interaction and I have a continuous outcome variable. However, I hypothesize that the DiD estimator can be moderated ...
- 1
1
vote
1
answer
54
views
Exercises for A/B testing/Experimental design
I'm applying for a data scientist job, which requires a lot of knowledge in A/B testing/Experimental design. But actually I'm not very familiar with it.
For A/B testing I was recommended a great book ...
0
votes
0
answers
22
views
Two way Mixed anova or something else ? (small sample size)
I would like to ask some questions about appropriate statistical test for our study design.
This study aims to evaluate the postural balance among parkinsonian patients.
We have 3 groups, and in each ...
0
votes
0
answers
29
views
How do I use Lehr's 'rule of thumb' with a binomial distribution?
(This is a crosspost from data science stack exchange: https://datascience.stackexchange.com/q/113436/139117)
Not a data scientist, but hopefully this is the right forum for this simple question: I'm ...
- 1
0
votes
0
answers
29
views
Minimum sample size for AB testing on a proportion derived from an extremely rare event
I am running an online experiment on a proportion that is the rate of an extremely rare event. 95.5% of the observations have 0 for this event. The majority of the rest have 1 for that event. Hence ...
- 76
0
votes
1
answer
45
views
Experiment design useful when objects change / will be replaced in each trial run?
I want to perform some studies on how material thickness, type of material, humidity and temperature influence tensile strength.
While I was thinking about useful designs I was wondering whether a ...
- 2,441
3
votes
1
answer
60
views
How to account learning effect in study which cannot be conducted simultaneously?
Suppose I want to conduct a trial with two treatment arms in each of which there are 50 patients. The patients in both arms are iid sample from sick population. I would like to compare treatment ...
- 1,179
1
vote
1
answer
33
views
A/B experimental design with strata or blocks?
I have two approaches I wish to evaluate conversions (currently at ~2%). As in many situations, I cannot assume equal likelihood of conversions of instances which any number of individuals can be ...
- 4,743
1
vote
1
answer
23
views
Adjust results for a systematic error
My model identifies objects with a specific quality in a large population. I tested it on a sample of 500 and found that it fails to identify about 7% of the objects (false negatives) due to some ...
- 41
2
votes
1
answer
49
views
Multiplicity issues in clinical trials for multiple groups
Let's say we design a RCT for two different populations and want to test an intervention vs a control. It could be adults and children, male and female, etc.
We have one primary endpoint to be tested ...
- 221
0
votes
0
answers
25
views
How should one re-formulate hypothesis after hypothesis testing or whether or not to discard data after significance testing or archive it?
@Nalzook summarized what he thinks Ronald Fisher did.
Ask a question.
Propose a null hypothesis based on the question.
Do some experiments, and collect some data.
Assuming the null is true, calculate ...
- 792
0
votes
0
answers
13
views
Name for expeimental design using randomization without replacement?
I have a between subjects design with 3 conditions A, B and C. My goal is to have 30 people in each condition. I have no other balance requirements other than the sample size being equal across ...
- 415
0
votes
0
answers
11
views
How are the interactions between three-level factor and four-level factor calculated?
I would like to know how the interactions between three-level factor and four-level factor can be calculated.
The textbook I have says how to calculate the following interactions.
between three-level ...
- 1
0
votes
0
answers
21
views
In linear regression, when sample size of one treatment group decreases, the power for testing the other treatment group vs common control also drop
Suppose we have design where we have three groups: two treatment groups and a common control. Suppose that when each of the three groups have sample size being 30, the power for testing whether ...
- 983
1
vote
0
answers
33
views
Mixture desing experiment and regression
I have an experiment with four groups with multiple trials. In two groups individuals have the exact same trials during all the experiment, no variation In the stimuli were added. But in the other ...
