Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [experiment-design]

The design of experiments (DOE) is the study of how to structure an information-gathering exercise where variation is present. In statistics, controlled experiments are usually implied, meaning that study units are randomly assigned to conditions where levels of a variable have been independently manipulated (prototypically into treatment & control groups).

0
votes
1answer
11 views

Difference between “distractors” and “foils”

What is the difference between "distractors" and "foils" in experimental design in psychology?
0
votes
0answers
11 views

Can't sample all conditions for each participant — strategies to compensate?

I am attempting to design a study using Category Comparison Rating / Double Stimulus Comparison Scale to evaluate participant perception of different stimulus conditions. The original study design ...
1
vote
0answers
13 views

Multi-level factor design of experiments

How should I select interactions in order to be able to encode multi-level features using interactions of main features in my design? I'm wanting to create an experimental design to be used to create ...
0
votes
1answer
22 views

Design of Experiments - Minimum Number of Experiments

I am new to experimental design. I am currently trying to determine the minimum number of trials required to estimate a linear regression model with one response variable and three predictive ...
0
votes
0answers
13 views

Potential problem in using paired t-test or Wilcoxon if the two samples consist of multiple draws from same group?

I have an experiment that works loosely works as follows: There are ten subjects in each treatment. There are two treatments, A and B. Each treatment has 8 rounds. In each round, each subject is ...
0
votes
0answers
4 views

Moderated mediation with Hayes PROCESS

Is there a way of using Hayes PROCESS for a mediation analysis, if I have three interacting categorical independent variables (with two levels each), 1 mediator (continuous) and 1 outcome variable (...
0
votes
0answers
16 views

Interpreting ANOVA results in the educational context

I have run an experiment in a six-week course to measure the effect of my educational treatment on student engagement. The treatment happens on weeks 4 and 6 of the course (for all of the students) ...
1
vote
1answer
18 views

Need help determining sample size for an AB test with two metrics

I need to run an experiment with customer support agents who are rated on two metrics by the customer: average issue resolution rate (proportion of issues resolved) and average star rating (1-5). ...
0
votes
0answers
8 views

Non replicated vs replicated field trial

I need to conduct a pesticide efficacy trial on avocados. I have been asked to describe two different trial approaches: replicated and non replicated. I have drawn the two trial design options. ...
0
votes
1answer
11 views

Explanation of Advantages of Balanced Designs with Statistical Theory

I wanted to know what the difference between balanced and unbalanced design. Firstly, here are two advantages of balanced designs: ...
0
votes
1answer
22 views

Sample Size Experiment Proportion Failure

I have N machines and I want to implement an experiment E in them in order to improve their proportion of failures. However, I’m not sure yet if this experiment E will increase, decrease or do ...
1
vote
1answer
26 views

What statistical analysis should I use for comparing multiple variables at different time frames? (psychology experiment)

I'm planning a longitudinal psychology experiment where I want to compare levels of different variables over the course of an 8-week treatment. i.e. I want to see any increase/decrease in variable A, ...
1
vote
1answer
24 views

How should I test a function which gives a measure of how well objects fit a category?

I am creating a function that gives a measure of how well a given object fits a specific category. The function is intended to give a measure that predicts how well in general humans think the object ...
1
vote
0answers
7 views

factor of safety

I need to validate a specification on a part. The requirement is to withstand a minimum 1 kg of force (i.e. not fail at less than 1 kg). I was told that if I measure, on 6 specimens, a force of 2.5 ...
-1
votes
0answers
19 views

Evaluating results from control group testing

In the classic case of target and control groups, the target and control group receive a differential treatment and the effect of this treatment is evaluated. This assumes that every member within the ...
0
votes
0answers
6 views

Counter balancing within-subject design or simply shuffling orders?

I have 4 treatments that are run on a set of 3 different game apps. That means every participant has to play 12 different combinations of game and treatment. Due to my limited number of participants (...
4
votes
0answers
25 views

conditional and interventional expectation

Conditional expectation $E[Y|X]$ and interventional expectation $E[Y|do(X)]$ are related but conceptually very different things. I know that if $X$ is a randomly assigned by an experiment, we have ...
1
vote
1answer
20 views

sample size calculation in 3-arm survival analysis

In a 3-arms clinical trial, with time to event data, either A is superior to C or B is superior to C it will be considered significant. No need to compare A and B. I know how to calculate the sample ...
1
vote
0answers
16 views

Measuring incremental revenue / uplift

I run a campaign with an experimental design setup. The target and control group is representative and split 70/30. The target group receives an email communication The dynamics of the campaign ...
0
votes
0answers
8 views

Variance of interblock estimator in balanced incomplete block design

I am working in a balanced incomplete block design, whose model is as follows: $$y_{ij} = \mu + \tau_i + \beta_j + \epsilon_{ij}, i = 1\ldots a, j = 1 \ldots b$$ where $y_{ij}$ is the value of the ...
2
votes
0answers
35 views

4 x 4 x 4 Factorial Design or 4 x 4 Factorial with Blocking?

Problem here: https://imgur.com/Nl8AWie I was thinking that this problem might just be a 3-factor factorial design, since obviously the ad campaign and packaging are factors (we're interested in ...
4
votes
1answer
32 views

Crew selection: ranking rowers by letting them race against each other

Seat selection is a common practice in competitive rowing and I would be curious about more solid statistical underpinnings: there are more rowers in a team than the 8 seats in the crew boat. So the ...
0
votes
1answer
70 views

Hypothesis testing control vs treatment when the outcome tends to vary over time

I want to test the effect of a treatment on a continuous outcome (specifically interested in increase in average in the treatment group). The experiment is run over a period of time. But the problem ...
0
votes
1answer
23 views

Modeling interaction in variance

I run an experiment on both men and women participants, where I manipulate an independent variable. I know that the manipulation causes greater variance in the outcomes of the experimental group (a ...
0
votes
0answers
16 views

Is ATE = 1/2 ATT + 1/2 ATC?

Is this statement true or false? Please provide justification. The following identity generally holds: ATE = 1/2 ATT + 1/2 ATC, where ATE is the average treatment effect, ATT is the average treatment ...
1
vote
0answers
16 views

Get estimates about 'variable importance' across a large number of variables and their permutations

My client would like to know which, of many different kinds of product dimensions, are most important for perceived quality. Some of the dimensions are as follows: price, material, country of origin, ...
0
votes
1answer
29 views

Why does unconfoundedness not hold if the covariates are affected by the treatment assignment?

I found a mention inside the Rubin-Imbens Causal Inference book that: Unconfoundedness is generally violated if X includes variables that are themselves affected by the treatment. For example, in ...
0
votes
0answers
13 views

How to test the validity of Fitt's law in R

I would like to use the Fitt's Law to test some hypothesis. Anyway, it's the first time I use it, and I have some doubts. Then aim of my experiment is to test if in a certain situation the Fitt's law ...
0
votes
0answers
9 views

REPEATED MEASURES DESIGN

I am not sure how the results of this study should be analysed given its design. I have searched everywhere for this info on this TYPE of experimental design and have had no luck. I don't even know ...
1
vote
0answers
22 views

G efficiency in Optimal designs

I am studying Optimal Designs and found a very interesting article by Peter Goos. In the article he provides an example in the form of an Excel document of generation of Optimal Designs for various ...
0
votes
0answers
18 views

Difference between Confounder, Mediator, Precision Variable, Effect Modifier

While I do have the definitions for each of these 'memorized', I find it hard to actually distinguish between them during the initial steps of confirmatory data analysis. Example Looking at how ...
1
vote
0answers
21 views

Experimental Design - Purely Experimental Variance

I am new to Experimental Design. I would greatly appreciate if you could help. Suppose one is interested in estimating a regression model with three explanatory variables (two of which are ...
0
votes
0answers
27 views

How to design an experiment to test the effect of a drug on subjective energy level

I am currently running an experiment to test the effect of a drug (ALCAR) on my energy level (measured using a subjective 5-point scale). My goal is to determine whether or not consuming this drug ...
0
votes
0answers
20 views

Statistical technique to assess change in drug events over time - Interrupted time series analysis?

I have a dataset of the number of drug related events (e.g. arrest, non-fatal overdoses) a population of individuals have had. My aim is to see if a certain policy intervention that occurred on 1 June ...
0
votes
0answers
18 views

How do I conduct an ANCOVA within a pre and posttest design?

I have a question regarding the analysis of my quasi field experiment. I conducted an experiment in a high school where a class learns over 5 weeks with a smart personal assistant (n=22) and the other ...
0
votes
0answers
22 views

How does Gaussian process update after adding a new point to the model?

I am using a Gaussian Process model in the Bayesian Optimization setting. Concretely, a gaussian process is built on some initial $N$ points and the model is updated sequentially by evaluating a new ...
1
vote
0answers
7 views

Question about books answer to estimating variance of block

The question is, Suppose that a single factor experiment with four factors and six blocks is conducted and the error sum of squares is 500 and the sum of squares for the blocks is 250. The question ...
1
vote
2answers
33 views

Is DoE applicable to collect data for machine learning model?

I'm currently working on a machine learning model for a classification task in an engineering application. While working on this project I realized that the provided data is insufficient to get a ...
0
votes
0answers
17 views

Why do we use this confidence interval in design of experiments

Consider the basic model, where $i=1,2,...a$ and $j=1,2...n$ $y_{ij}=\mu+\tau_{i}+\epsilon_{ij}$ Where $\epsilon_{ij} \sim N(0,\sigma^{2})$ and $var(\tau_{i})= \sigma_{\tau}^{2}$ and $cov(y_{ij},y_{...
0
votes
0answers
37 views

Hypothesis testing with 1 or 2 sample in ornithology

I am doing an experiment with birds. I have set up the experiment to measure how certain foods affect bird flying speeds. When I implement a control, for example food A, I want to claim that the ...
2
votes
1answer
56 views

ANOVA with Error term

I am looking for help to create a model for the following data: I'd like to understand if the abundance of a certain microbial group (dependent continuous variable QUANTITY) found in soils is been ...
1
vote
1answer
55 views

Why do we do crossed vs. nested vs. other random effects?

Let's try a theoretical example. I am trying to predict the math scores of students within schools. I see three ways I can model this with random effects: (1) I can "nest" the random effects. My ...
0
votes
2answers
38 views

Multiple comparison problem

If I conduct 19 tests (e.g. 19 pairwise comparisons of treatment means in an experiment), what is the probability that I will find at least one P value < 0.05 due to chance alone? With number of ...
1
vote
1answer
29 views

Difference between N case studies and case series with N patients

Is there any reason to treat data from multiple case studies differently from data from a case series that reports data on multiple patients? Assuming the case-series intervention was the same for ...
0
votes
0answers
10 views

quasi-experiment

I am doing one group pre and post test experiment. my samples will be selected by the hotel managers. so , it is random sampling. my question is, can I do a one group pre and post test experiment with ...
0
votes
1answer
28 views

Factor importance in multivariate experiment

I'm trying to analyze a multivariate experiment and show impact of each factor in a multivariate experiment. The general way to do this analysis would be to perform ANOVA: ...
0
votes
0answers
31 views

Experimental Design Fixed Effect Minimal Detectable Difference

How does one determine the minimal detectable difference for a fixed factor/interaction in a general balanced experimental design? For example, I would like to be able to recreate Appendix 12 of ...
0
votes
0answers
35 views

Difference-in-Differences (DID) estimator with fertility data

In a nutshell, I'm using DID to estimate the impact of a cash-transfer program on fertility in a specific country. The program offers families an untaxed, unconditional cash transfer for every second ...
2
votes
0answers
28 views

How do you design a regression experiment with many variables but samples are expensive?

I want to design an experiment to see which factors affect production quality. Quality can either be a binary response like "good" or "bad" or a quantitative value like noise level. There are 10 ...
1
vote
1answer
22 views

Comparing one group to many others

I have a quasi-experiment where I'm comparing one experimental group (A) to many control groups (B1, B2, B3, B4, B5). Since this is a quazi-experiment, I do not know for sure that my control groups ...