Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Create the same consistant optimized DoE with optFederov() [closed]

When I create a full factorial plan with the fac.design() function from the DoE.base package and I need to reduce the number of necessary runs, the ...
  • 1
0 votes
0 answers
5 views

Inter-rater agreement for multiple factors for multiple pictures?

As part of my Master's dissertation, I am using photos and narratives to elicit a specific emotion. I have had 12 photos rated on various aspects of the emotion e.g. valence, arousal etc. I need ...
0 votes
0 answers
8 views

Analyzing data from a mixed experiment (nested or crossed?)

I am trying to analyze data from an experiment where my collaborators and I were interested in whether tweets about risk can influence people's risk perception (a continuous dependent variable). We ...
2 votes
0 answers
28 views

Draw Maximum Information from Experts to set up Utility Function

We aim to draw maximum information from CEP-logistics experts to set up a utility function for delivery chain choices for each parcel. The choices / dependent variables are specific transport chains, ...
  • 31
0 votes
0 answers
34 views
+50

Are fractional factorial designs a part of 'Response Surface Methodology' or something different?

Fractional factorial designs (FFDs) and Response Surface Methodology (RSM) are both approaches to extracting some information about how multiple interacting factors affect some response variable. ...
  • 15k
3 votes
0 answers
61 views
+50

Practical power analysis on linear model

Power analysis for linear model test asks for an analytical expression of the power of a linear model. The answer claims that Note that this depends on the correlation between $X$ and $T$, $r_{TX}$. ...
0 votes
0 answers
7 views

Experiment terminology: control participants invertedly receive treatment

I have spent a solid hour to remember with the technical term for this. Suppose you have a between-subjects lab design with one control and two treatment conditions. What is the term to describe when ...
0 votes
0 answers
18 views

Split-split-plot design: Analysis and error terms

Can I have three levels of randomization and three error terms even when the replications are produced under same whole plot treatments under split split plot design? For example, three reps are ...
0 votes
0 answers
21 views

Do I need to use Latin Square when I only have one variable with two levels?

I am currently designing an acceptability judgment experiment that contains one variable with two levels. The experiment will be a within-subject design. There will be multiple sets of target ...
  • 101
0 votes
0 answers
24 views

Is there a function in R to give the lack of fit and pure error of an ANOVA completed on a half 2^k factorial with centre points?

I have a half fraction factorial experiment for four factors, meaning I have eight experiments that I conduct. I expect that some of the responses are quadratic and not linear, thus I included centre ...
1 vote
1 answer
28 views

A/B test with two metrics, we test one metrics for improvement on the condition the other one doesn't change

I have the following business request: "Test this change on the landing page and implement it for all the visitors if the test shows improvement in the conversion rate, but if the page load speed ...
7 votes
4 answers
754 views

Why is the Central Limit applicable in A/B testing?

I am having trouble understanding why the Central Limit Theorem (CLT) is applicable in A/B testing. As a beginner in statistics, I am trying to grasp the intuition behind it. The CLT states that as we ...
  • 71
0 votes
0 answers
11 views

Best DV operationalization for statistical power

I am setting up a questionnaire for a lab experiment to measure support for four competing policies. Very importantly, I want to know the public's order of preference. I am torn on how best to ...
0 votes
0 answers
19 views

Comparing treatment effects across groups

I created two donation campaigns, one loss framed (N:1993) and one gained framed (N:1989) I understood donations are usually veeeery low and the sample was a lot smaller than previously expected, so ...
  • 3
2 votes
2 answers
46 views

Design of a Dental RCT in Which Patients Can Act as Their Own Control

I've been asked to help determine the design of an RCT. The design is primarily interested in the benefit of using a laser on some dental procedure in addition to a local anti-biotic and scaling, ...
0 votes
0 answers
16 views

Permutation design for repeated measures via how() in RDA

We would like to properly set up the permutation design in anova() testing for our rda(). Study design We chose several areas, ...
  • 1
0 votes
0 answers
20 views

What's the null hypothesis of A/B test following a sample size calculation?

I have some confusion around what the null hypothesis should be for an A/B test. When we decide to do an A/B test and proceed to perform a sample size calculation. Let's say we work with the following ...
  • 23
1 vote
0 answers
11 views

Does this experimental design avoid pseudo-replication?

I’m looking at a data set where 15 properties undertaking new management practices are sampled over three seasons. Measurements taken include variables such as vegetation cover and canopy cover. ...
0 votes
1 answer
40 views

A/B test - sample size calculation and unit of measurement

Does the unit of measurement change how we calculate sample size? For example, if we were to run an A/B test on ads with our target metric being click-through-rate, and we are doing a sample size ...
  • 51
0 votes
1 answer
36 views

A/B Testing Proportions - baseline rate is close to 1

I need a gut check here... I'm trying to run an A/B test where the baseline metric is a conversion rate that is close to 1 - somewhere in the 80-85% range. And so when I run my sample size calculation ...
  • 51
0 votes
0 answers
13 views

Can we do some causal conclusions if the mediator follows the dependent variable?

We have a survey consisting of two waves ($N=4000$). In Wave 1, we ask block of questions $M_1$. In Wave 2, we ask blocks of questions $W$, $Y$, $M_2$ The order of questions is as follows: $W \...
0 votes
1 answer
60 views

How to determine sample size for non-normal distributed data with continuous variables?

TLDR: I need to determine the minimum sample size for measuring the energy consumption of code. Hello, I'm trying to determine how many samples (N) I need in an experiment to be confident that my ...
0 votes
1 answer
47 views

A/B test: measure likelihood of outcomes based on historical events

I am researching if it makes sense to create personalized emails based on past purchase behavior, but I do not know how to design A/B tests. The starting data I have is a list of about 500,000 ...
0 votes
0 answers
23 views

Random effects structure for logistic regression for RCBD with subsamples in glmer

I have a dataset that consists of a randomized complete block design with 4 blocks and 4 treatments. The response is a binary response of survival from approximately 80 subsamples per treatment. My ...
1 vote
1 answer
23 views

Experimental design with two factorial treatment and blocks of the treatment combination

I am going to do an experiment with planting seeds involving 2 factorial treatments. There are 4 families of seeds: F1, F2, F3, F4 and 3 foliar seed treatments: T1, T2, T3. There will be 160 seeds for ...
0 votes
0 answers
31 views

What is a principle of least perturbations?

Background In BPF Performance Tools by Brendan Gregg, there is a section describing a series of 18 experiments measuring the overhead of tracing with eBPF on Linux. In the description of the ...
1 vote
0 answers
15 views

How to combine within-subject score for between analysis when there is no within ID

I was task with analyzing some within/between-subject data, but my current guess is that I have no way of analyzing a combined within-subject score since an identifier/ID is missing. Is there a ...
  • 11
1 vote
0 answers
74 views

Sample Size for categorical variables with given accuracy

I am trying to do a design of experiment to detect a defect (with little information given to me). In order to do that, I need to calculate the sample size. I am not really sure whether my attempt is ...
  • 11
2 votes
2 answers
199 views

Model Sum of Squares from ANOVA table

Background: I am studying a course on statistical experiments using the textbook by Douglas Montgomery on the analysis of experiments. This is an introductory course and so I am relatively new to the ...
  • 173
0 votes
0 answers
26 views

T-Test for Research Project (High School Student)?

I am completing a research project on how Heart Rate is affected by wearing two different types of shoes during 4 minute trials at VO2 Max (Reaching close to Max HR). One shoe is the Nike Vaporfly ...
1 vote
0 answers
32 views

Multilevel Bayesian model for Experimental Design with levels in treatment and control groups

Context I'm designing an experiment and Bayesian analysis to interpret results. I've used Statistical Rethinking as inspiration for model structure. Suppose I want to study the effects of a sports ...
  • 2,123
1 vote
1 answer
70 views

Least Squares Estimation for CRD experiments

Setup Let us use the following model for a Completely Randomised Designed (CRD) experiment: $$ Y_{ij}= \mu + \tau_i + \epsilon_{ij}$$ where the errors $\epsilon _{ij}$ ~ Normal ($0, \sigma ^2 $) and ...
  • 173
1 vote
1 answer
52 views

Understanding the advantages of CRD experiments

I am self studying an introductory course on Designed Experiments and have come across the notion of a Completely Randomised Design (CRD) defined as follows: Completely Randomised Design: the ...
  • 173
0 votes
0 answers
22 views

Difference-in-Difference: What to do with control observations when there are multiple possible intervention periods

I have several years worth of individual level data on several thousand individuals and a corresponding measure. During those years several hundred individuals underwent an intervention with the hope ...
6 votes
1 answer
371 views

Interpretation of causal effect from instrumental variables

We ran an experiment in which users could optionally watch a video. The video is intended to drive some other metric, $y$. Because the intervention (video) was optional, choosing to watch the video ...
0 votes
0 answers
16 views

BACI analysis where impact variable is continuous

I am analysing data from a before-after control-impact (BACI) experiment where all 10 sites (replicates) were measured once before a disturbance, then 5 sites were subjected to the disturbance, and ...
2 votes
0 answers
88 views

stepAIC in mixed models

As I haven't found the equivelant of the MASS::stepAIC for mixed models (eg in lmer) what I'm intending to do is to find the best lm model using stepAIC and then go in lmer and add the random effects. ...
  • 295
5 votes
1 answer
140 views

Experimental design for a 96 well plate

I am designing an experiment that will include an assay on a 96 well plate and could use some advice on the optimal layout. In our experiment we have 12 biological replicates, and each replicate has ...
  • 1,192
2 votes
1 answer
93 views

What are balanced and orthogonal designed experiments?

I have recently started studying Designed Experiments and have have come across non-rigorous definitions of "balanced" and "orthogonal" experiments and would be interested in ...
  • 173
0 votes
1 answer
130 views

Relationship between blocks, factors and treatments

I have recently began studying a course on Designed Experiments and am having some trouble understanding some of the terminology. I've looked at some other answers on the site and I think that I am ...
  • 173
1 vote
0 answers
20 views

Surrogate model data requirements

I am trying to undersand how to approach the development of a surrogate model to investigate the associated uncertainty when predicting late arrival times. I.e there is an, to me, agnostic and ...
  • 229
1 vote
0 answers
32 views

Between subject design, spread over multiple days

We have a between subjects design, testing for the impact of aesthetics on occupant satisfaction. Intervention is carpets vs no carpets. The people participating in each leg (intervention vs control) ...
0 votes
1 answer
15 views

How much of a discrepency is tolerable if Sample Ratio Mismatch is detected?

We have analyzed several past online A/B tests and revealed that some of them have SRM at a statistically significant level (p<0.001). Example: we had 50%/50% traffic allocation between control and ...
1 vote
0 answers
24 views

Treatment subspaces in Bailey's Design of Comparative Experiments

A draft of Bailey's Design of Comparative Experiments may be found here; I am referring to Chapter 2, figure 2.1. Consider $t$ treatments. We have $N$ plots labeled $1, \dots, N$. We apply treatment $...
  • 4,153
1 vote
0 answers
8 views

Estimating required sample size in cases when the base conversion rate always heavily fluctuates

I run marketing experiments on a website. As far as I know, it's a well-known rule that we should estimate required sample size to detect a certain uplift in the treatment before running the ...
0 votes
0 answers
63 views

Calculating Sample Sizes for Tukey and Scheffe Pairwise Contrasts

$\newcommand{\x}[1]{\text{#1}}\newcommand{\op}[1]{\operatorname{#1}}$ This is Exercise 4.9 of Design and Analysis of Experiments, 2nd Ed., by Dean, Voss, and Draguljic. Problem Statement: Suppose the ...
0 votes
0 answers
25 views

Clustering zip data at state level

Problem : Recomment 2 markets (States) for A/B test Solution tried : Cluster rows based simply by ZIP codes but then each state could appear across multiple clusters. How do I then dedup the results?...
1 vote
1 answer
53 views

Comparing total spent in unequal groups (lognormal distributed)

I have 100K customers from a population with a KPI name "spent" which is lognormal distributed. The customers were split into 2 groups - 1st group - customers with an id ending with 0 - ...
2 votes
1 answer
38 views

How to decide time duration for synthetic control

For A/B test, one can use power analysis to determine the duration of experiment to run. What about synthetic control method? Is there a way to determine the duration of experiments post intervention?
1 vote
0 answers
33 views

Fractional factorial design with mixed categorical and numerical variables analysis for more than two levels

I have an experiment setup that consists of multiple continuous and multiple categorical variables. Right now, I am just using two levels for the categorical variables, allowing me to encode them as -...
  • 111

1
2 3 4 5
38