Questions tagged [experiment-design]
The design of experiments (DOE) is the study of how to structure an information-gathering exercise where variation is present. In statistics, controlled experiments are usually implied, meaning that study units are randomly assigned to conditions where levels of a variable have been independently manipulated (prototypically into treatment & control groups).
0
votes
1answer
11 views
Difference between “distractors” and “foils”
What is the difference between "distractors" and "foils" in experimental design in psychology?
0
votes
0answers
11 views
Can't sample all conditions for each participant — strategies to compensate?
I am attempting to design a study using Category Comparison Rating / Double Stimulus Comparison Scale to evaluate participant perception of different stimulus conditions. The original study design ...
1
vote
0answers
13 views
Multi-level factor design of experiments
How should I select interactions in order to be able to encode multi-level features using interactions of main features in my design?
I'm wanting to create an experimental design to be used to create ...
0
votes
1answer
22 views
Design of Experiments - Minimum Number of Experiments
I am new to experimental design. I am currently trying to determine the minimum number of trials required to estimate a linear regression model with one response variable and three predictive ...
0
votes
0answers
13 views
Potential problem in using paired t-test or Wilcoxon if the two samples consist of multiple draws from same group?
I have an experiment that works loosely works as follows:
There are ten subjects in each treatment. There are two treatments, A and B. Each treatment has 8 rounds. In each round, each subject is ...
0
votes
0answers
4 views
Moderated mediation with Hayes PROCESS
Is there a way of using Hayes PROCESS for a mediation analysis, if I have three interacting categorical independent variables (with two levels each), 1 mediator (continuous) and 1 outcome variable (...
0
votes
0answers
16 views
Interpreting ANOVA results in the educational context
I have run an experiment in a six-week course to measure the effect of my educational treatment on student engagement. The treatment happens on weeks 4 and 6 of the course (for all of the students) ...
1
vote
1answer
18 views
Need help determining sample size for an AB test with two metrics
I need to run an experiment with customer support agents who are rated on two metrics by the customer: average issue resolution rate (proportion of issues resolved) and average star rating (1-5). ...
0
votes
0answers
8 views
Non replicated vs replicated field trial
I need to conduct a pesticide efficacy trial on avocados. I have been asked to describe two different trial approaches: replicated and non replicated.
I have drawn the two trial design options. ...
0
votes
1answer
11 views
Explanation of Advantages of Balanced Designs with Statistical Theory
I wanted to know what the difference between balanced and unbalanced design.
Firstly, here are two advantages of balanced designs:
...
0
votes
1answer
22 views
Sample Size Experiment Proportion Failure
I have N machines and I want to implement an experiment E in them in order to improve their proportion of failures.
However, I’m not sure yet if this experiment E will increase, decrease or do ...
1
vote
1answer
26 views
What statistical analysis should I use for comparing multiple variables at different time frames? (psychology experiment)
I'm planning a longitudinal psychology experiment where I want to compare levels of different variables over the course of an 8-week treatment. i.e. I want to see any increase/decrease in variable A, ...
1
vote
1answer
24 views
How should I test a function which gives a measure of how well objects fit a category?
I am creating a function that gives a measure of how well a given object fits a specific category. The function is intended to give a measure that predicts how well in general humans think the object ...
1
vote
0answers
7 views
factor of safety
I need to validate a specification on a part.
The requirement is to withstand a minimum 1 kg of force (i.e. not fail at less than 1 kg).
I was told that if I measure, on 6 specimens, a force of 2.5 ...
-1
votes
0answers
19 views
Evaluating results from control group testing
In the classic case of target and control groups, the target and control group receive a differential treatment and the effect of this treatment is evaluated. This assumes that every member within the ...
0
votes
0answers
6 views
Counter balancing within-subject design or simply shuffling orders?
I have 4 treatments that are run on a set of 3 different game apps. That means every participant has to play 12 different combinations of game and treatment.
Due to my limited number of participants (...
4
votes
0answers
25 views
conditional and interventional expectation
Conditional expectation $E[Y|X]$ and interventional expectation $E[Y|do(X)]$ are related but conceptually very different things.
I know that if $X$ is a randomly assigned by an experiment, we have ...
1
vote
1answer
20 views
sample size calculation in 3-arm survival analysis
In a 3-arms clinical trial, with time to event data, either A is superior to C or B is superior to C it will be considered significant. No need to compare A and B.
I know how to calculate the sample ...
1
vote
0answers
16 views
Measuring incremental revenue / uplift
I run a campaign with an experimental design setup. The target and control group is representative and split 70/30.
The target group receives an email communication
The dynamics of the campaign ...
0
votes
0answers
8 views
Variance of interblock estimator in balanced incomplete block design
I am working in a balanced incomplete block design, whose model is as follows: $$y_{ij} = \mu + \tau_i + \beta_j + \epsilon_{ij}, i = 1\ldots a, j = 1 \ldots b$$ where $y_{ij}$ is the value of the ...
2
votes
0answers
35 views
4 x 4 x 4 Factorial Design or 4 x 4 Factorial with Blocking?
Problem here: https://imgur.com/Nl8AWie
I was thinking that this problem might just be a 3-factor factorial design, since obviously the ad campaign and packaging are factors (we're interested in ...
4
votes
1answer
32 views
Crew selection: ranking rowers by letting them race against each other
Seat selection is a common practice in competitive rowing and I would be curious about more solid statistical underpinnings: there are more rowers in a team than the 8 seats in the crew boat. So the ...
0
votes
1answer
70 views
Hypothesis testing control vs treatment when the outcome tends to vary over time
I want to test the effect of a treatment on a continuous outcome (specifically interested in increase in average in the treatment group). The experiment is run over a period of time. But the problem ...
0
votes
1answer
23 views
Modeling interaction in variance
I run an experiment on both men and women participants, where I manipulate an independent variable. I know that the manipulation causes greater variance in the outcomes of the experimental group (a ...
0
votes
0answers
16 views
Is ATE = 1/2 ATT + 1/2 ATC?
Is this statement true or false? Please provide justification.
The following identity generally holds: ATE = 1/2 ATT + 1/2 ATC, where ATE is the average treatment effect, ATT is the average treatment ...
1
vote
0answers
16 views
Get estimates about 'variable importance' across a large number of variables and their permutations
My client would like to know which, of many different kinds of product dimensions, are most important for perceived quality. Some of the dimensions are as follows: price, material, country of origin, ...
0
votes
1answer
29 views
Why does unconfoundedness not hold if the covariates are affected by the treatment assignment?
I found a mention inside the Rubin-Imbens Causal Inference book that:
Unconfoundedness is generally violated if X includes variables that
are themselves affected by the treatment. For example, in ...
0
votes
0answers
13 views
How to test the validity of Fitt's law in R
I would like to use the Fitt's Law to test some hypothesis. Anyway, it's the first time I use it, and I have some doubts.
Then aim of my experiment is to test if in a certain situation the Fitt's law ...
0
votes
0answers
9 views
REPEATED MEASURES DESIGN
I am not sure how the results of this study should be analysed given its design.
I have searched everywhere for this info on this TYPE of experimental design and have had no luck. I don't even know ...
1
vote
0answers
22 views
G efficiency in Optimal designs
I am studying Optimal Designs and found a very interesting article by Peter Goos. In the article he provides an example in the form of an Excel document of generation of Optimal Designs for various ...
0
votes
0answers
18 views
Difference between Confounder, Mediator, Precision Variable, Effect Modifier
While I do have the definitions for each of these 'memorized', I find it hard to actually distinguish between them during the initial steps of confirmatory data analysis.
Example
Looking at how ...
1
vote
0answers
21 views
Experimental Design - Purely Experimental Variance
I am new to Experimental Design. I would greatly appreciate if you could help.
Suppose one is interested in estimating a regression model with three
explanatory variables (two of which are ...
0
votes
0answers
27 views
How to design an experiment to test the effect of a drug on subjective energy level
I am currently running an experiment to test the effect of a drug (ALCAR) on my energy level (measured using a subjective 5-point scale). My goal is to determine whether or not consuming this drug ...
0
votes
0answers
20 views
Statistical technique to assess change in drug events over time - Interrupted time series analysis?
I have a dataset of the number of drug related events (e.g. arrest, non-fatal overdoses) a population of individuals have had. My aim is to see if a certain policy intervention that occurred on 1 June ...
0
votes
0answers
18 views
How do I conduct an ANCOVA within a pre and posttest design?
I have a question regarding the analysis of my quasi field experiment. I conducted an experiment in a high school where a class learns over 5 weeks with a smart personal assistant (n=22) and the other ...
0
votes
0answers
22 views
How does Gaussian process update after adding a new point to the model?
I am using a Gaussian Process model in the Bayesian Optimization setting. Concretely, a gaussian process is built on some initial $N$ points and the model is updated sequentially by evaluating a new ...
1
vote
0answers
7 views
Question about books answer to estimating variance of block
The question is,
Suppose that a single factor experiment with four factors and six blocks is conducted and the error sum of squares is 500 and the sum of squares for the blocks is 250. The question ...
1
vote
2answers
33 views
Is DoE applicable to collect data for machine learning model?
I'm currently working on a machine learning model for a classification task in an engineering application. While working on this project I realized that the provided data is insufficient to get a ...
0
votes
0answers
17 views
Why do we use this confidence interval in design of experiments
Consider the basic model, where $i=1,2,...a$ and $j=1,2...n$
$y_{ij}=\mu+\tau_{i}+\epsilon_{ij}$
Where $\epsilon_{ij} \sim N(0,\sigma^{2})$
and $var(\tau_{i})= \sigma_{\tau}^{2}$
and $cov(y_{ij},y_{...
0
votes
0answers
37 views
Hypothesis testing with 1 or 2 sample in ornithology
I am doing an experiment with birds. I have set up the experiment to measure how certain foods affect bird flying speeds. When I implement a control, for example food A, I want to claim that the ...
2
votes
1answer
56 views
ANOVA with Error term
I am looking for help to create a model for the following data:
I'd like to understand if the abundance of a certain microbial group (dependent continuous variable QUANTITY) found in soils is been ...
1
vote
1answer
55 views
Why do we do crossed vs. nested vs. other random effects?
Let's try a theoretical example. I am trying to predict the math scores of students within schools. I see three ways I can model this with random effects:
(1) I can "nest" the random effects. My ...
0
votes
2answers
38 views
Multiple comparison problem
If I conduct 19 tests (e.g. 19 pairwise comparisons of treatment means in an experiment), what is the probability that I will find at least one P value < 0.05 due to chance alone? With number of ...
1
vote
1answer
29 views
Difference between N case studies and case series with N patients
Is there any reason to treat data from multiple case studies differently from data from a case series that reports data on multiple patients?
Assuming the case-series intervention was the same for ...
0
votes
0answers
10 views
quasi-experiment
I am doing one group pre and post test experiment. my samples will be selected by the hotel managers. so , it is random sampling. my question is, can I do a one group pre and post test experiment with ...
0
votes
1answer
28 views
Factor importance in multivariate experiment
I'm trying to analyze a multivariate experiment and show impact of each factor in a multivariate experiment. The general way to do this analysis would be to perform ANOVA:
...
0
votes
0answers
31 views
Experimental Design Fixed Effect Minimal Detectable Difference
How does one determine the minimal detectable difference for a fixed factor/interaction in a general balanced experimental design?
For example, I would like to be able to recreate Appendix 12 of ...
0
votes
0answers
35 views
Difference-in-Differences (DID) estimator with fertility data
In a nutshell, I'm using DID to estimate the impact of a cash-transfer program on fertility in a specific country. The program offers families an untaxed, unconditional cash transfer for every second ...
2
votes
0answers
28 views
How do you design a regression experiment with many variables but samples are expensive?
I want to design an experiment to see which factors affect production quality. Quality can either be a binary response like "good" or "bad" or a quantitative value like noise level.
There are 10 ...
1
vote
1answer
22 views
Comparing one group to many others
I have a quasi-experiment where I'm comparing one experimental group (A) to many control groups (B1, B2, B3, B4, B5). Since this is a quazi-experiment, I do not know for sure that my control groups ...
