Questions tagged [experiment-design]
The design of experiments (DOE) is the study of how to structure an information-gathering exercise where variation is present. In statistics, controlled experiments are usually implied, meaning that study units are randomly assigned to conditions where levels of a variable have been independently manipulated (prototypically into treatment & control groups).
0
votes
0answers
7 views
Experimental Design Fixed Effect Minimal Detectable Difference
How does one determine the minimal detectable difference for a fixed factor/interaction in a general balanced experimental design?
For example, I would like to be able to recreate Appendix 12 of ...
0
votes
0answers
26 views
Difference-in-Differences (DID) estimator with fertility data
In a nutshell, I'm using DID to estimate the impact of a cash-transfer program on fertility in a specific country. The program offers families an untaxed, unconditional cash transfer for every second ...
2
votes
0answers
24 views
How do you design a regression experiment with many variables but samples are expensive?
I want to design an experiment to see which factors affect production quality. Quality can either be a binary response like "good" or "bad" or a quantitative value like noise level.
There are 10 ...
0
votes
1answer
15 views
Comparing one group to many others
I have a quasi-experiment where I'm comparing one experimental group (A) to many control groups (B1, B2, B3, B4, B5). Since this is a quazi-experiment, I do not know for sure that my control groups ...
0
votes
0answers
14 views
From Anova table to linear model coefficients
I understand how to construct an ANOVA table and compute the various Sums of Squares.
My query is : How do we go from Sums of Squares to the effect sizes ?
Can someone please explain ?
0
votes
0answers
25 views
Calculating power for balanced incomplete block design
My experiment has a factorial 2x2 design: 2 factors are manipulated at levels 'high' and 'low'. We thus get 4 conditions:
Condition 1: Factor A high, Factor B high
Condition 2: Factor A high, Factor ...
1
vote
0answers
22 views
Zeros in Experiment Analysis
I am analyzing data of an experiment close to the ultimatum game.
Participant A makes a decision x or y, and then Participant B can decide whether to punish Participant A. Participant B can choose ...
0
votes
1answer
30 views
Which experiment design to choose and when?
I went through a "design of experiment" course recently, but there are still things that I haven't sorted out yet regarding which types of experiment to choose when facing a real world problem.
I ...
0
votes
0answers
7 views
Weighted D optimality
I am working with D optimality for nonlinear models. My model has 8 parameters and some are more important than others. Is there a way to give weights to each of the parameters?
I am thinking of ...
0
votes
1answer
20 views
When to use blocks?
I am currently studying Random Complete Block Designs (RCBD) and I would like to ask a question.
I cam across an example where we were able to conclude with the RCBD as significant, but regular ANOVA ...
1
vote
0answers
26 views
Understanding Lenth Plot in R
I would like to understand how to interpret the Lenth Plot. From what I know from the LenthPlot R page, this plot displays the factor effects with significance levels. But I don't really get what a ...
0
votes
0answers
8 views
Taguchi Designs for differential gene expression
I plan to do a differential gene expression using RT-qPCR of 10 genes on 3 strains cultured in different enviormental conditions of 3 factors (2 levels each one). In order to do so, I want to follow a ...
0
votes
0answers
31 views
How to do a sensitivity analysis on a non-linear equation?
In the company, it is very difficult to actually do quotations for our customers properly because we do not have perfect information regarding the factors that affect the cost and profit. So I created ...
1
vote
1answer
42 views
Select optimal points for Gaussian process with a well-known target function
I'm currently trying to select the optimal points for a Gaussian Process Regression, and the important thing is that i already know the whole target function.
Therefore, it's not Online Learning ...
0
votes
2answers
29 views
Hypothetical experimental design - help needed for statistical analysis
I am proposing a hypothetical lab experiment where participants are randomly assigned into T(exposure to social norms messaging) and C (neutral messaging).
Each group is then subsequently asked their ...
0
votes
0answers
17 views
Setting an A/B test for electronic devices after an update
I want to set up an experimental testing or A/B test.
We have electronic devices on the field. Recently we had a problem with the keydoor. It started falling(it wont open(Fail) after we try to open ...
0
votes
0answers
10 views
Within-subjects experiment using a subset of conditions
I have an experimental design question for which I still haven't found an appropriate answer.
Briefly, I'm preparing a user study where we have 11 scenarios. Each scenario involves watching videos ...
0
votes
0answers
14 views
Represent Mean-Squared-Prediction error as function of covariance (or Fisher) matrix
Given a simple linear model:
$$
y_i = x_i^T \beta + \epsilon_i
$$
For simplicity, $\epsilon_i$ is Gaussian iid with variance $\sigma_e^2$, then the solution for $\hat{\beta}$ is given via Ordinary ...
0
votes
0answers
17 views
Hypothesis testing on set of comparisons experiment
Imagine I'm running an experiment where I'm comparing my method with $M-1$ competitors and I'm collecting the results for $T$ different runs. I would like to test the hypothesis of "my method is the ...
0
votes
0answers
7 views
What should be the layout of blocks in an agricultural CRBD experiment; by definition in a file or not?
I was having a discussion with a coworker concerning the layout of the blocks in an greenhouse experiment.
I assumed that the main purpose of blocks was best served if the variation between blocks ...
1
vote
0answers
42 views
How can we analyze a dataset with a pseudoreplication problem?
I am trying to analyse a data set with a pseudoreplication problem and would be grateful if someone could give me some guidelines on the best way to deal with it.
We have a set of subjects exposed to ...
0
votes
0answers
33 views
order of levels in crossed design
How can I determine if a factor A is nested in B or B is nested in ...
1
vote
0answers
32 views
Using a supervised learning to compare two conditions
I've got to analyze data about two signals x and y using machine learning but am stuck with how to proceed. Conditions are:
1) signal x and y are known to be linked to one another, but no parametric ...
0
votes
0answers
22 views
Product Packaging Problem: Measuring by pieces vs. Measuring by grams
I am helping a friend with his chicharon (pork cracklings) business. Initially, each selling package contains 50 grams of the product but he changed his mind and wanted 20 pieces instead for each ...
3
votes
1answer
76 views
AB test sample size calculation by hand
Evan Miller has created a well-known online AB test sample size calculator. For the sake of being able to program and modify this formula, I would like to know how to calculate sample size Evan Miller-...
1
vote
0answers
19 views
Power calculations for moderator analyses in RCT
I am running a randomised controlled trial of a savings intervention in India whereby randomisation is at the individual level (not clusters). In my power calculations, I also want to factor in a ...
0
votes
1answer
39 views
In marketing experiment, can we determine the effect customer state has on response?
We have the goal of determining whether an email campaign has an impact on the amount of product that customers purchase. We have customers from several states but much more from some states than ...
0
votes
0answers
27 views
Experimental design: two distinct, extreme group vs gradient design
I am designing an experiment in which I am exposing animals (the salt marsh-inhabiting amphipod Orchestia gammarellus) to environmental drought and salinity stress. I want to induce an evolutionary ...
3
votes
1answer
44 views
How to calculate (standardized) orthogonal contrast coding in R?
I want to determine efficient fractional factorial designs and blocked designs for factorial surveys and am using the R library AlgDesign. The criteria I'm using to determine if a design is efficient ...
0
votes
0answers
15 views
Experimental Design - daily surveys over the course of a month
I am trying to figure out the best way to design this experiment:
Each day for a month, a group of people is given a survey containing two questions: 1) What is your overall mood today? [6 point ...
0
votes
0answers
8 views
Estimating Effective Sample Size in Unequal clustered design (ratio-type estimator)
I'm having trouble understanding how these experimenters calculated their effective sample size.
The free article is located in this link here: https://academic.oup.com/icesjms/article/60/2/297/...
0
votes
0answers
21 views
defining arbitrary blocks
In an experiment with 4 treatments (a, b, c, d) and 3
observation per treatment, is it valid or desirable
to impose $n$ blocks on the design. That is
there is nothing that distinguishes the blocks.
...
0
votes
0answers
19 views
Proof that the pooled sample variance is biased when paired comparison is not used when it should be
Given the model for what is meant to be a paired comparison design:
$y_{ij} = \mu_i + \beta_j + \epsilon_{ij}$
$i = 1,2; j= 1, 2, ..., 10$
Eg. An experiment comparing the different in mean hardness ...
2
votes
1answer
19 views
Checking for group differences after randomly assigning subjects to one group or another?
If an experimenter assigns subjects to an experimental vs a control group truly randomly, is there still any point to performing baseline comparisons between the groups, after they've been so defined? ...
0
votes
1answer
21 views
Correlating perception with an objective measure - average ratings vs replicating the objective measure
We want to be able to predict user ratings of the appeal of some objects.
We have a hypothesized objective feature that may correlate with the user ratings. We want to test whether this objective ...
1
vote
1answer
34 views
Split split plot ANOVA table: r = replicate or block?
I have seen various ANOVA tables for analysis of split split plot designs, and some have the first term ''r'' defined as the block while others have it as the replicate. This is confusing to me, as ...
4
votes
1answer
32 views
Matching in a case control study
I'm working for a physician who thinks they would like to run a matched case control study, matching on 3 covariates, to assess the effect of exposure to a drug on an outcome. I've never done a case ...
0
votes
0answers
9 views
Usage of correlated (non-linear correlation) variables in an experiment and standardisation of variables values
The setting (see the dataset at the end of the question)
The setting of the problem is this:
I ask many multiple choice questions;
only 1 out of the 3 available answers is correct;
I ask the same ...
1
vote
2answers
54 views
Confounding variables in experimental study
We conducted a study to analyse the effect of tablet named 'xab' that help smokers to stop smoking. 5500 of smokers are selected. half of them were given different doses of tablet while the other ...
2
votes
1answer
34 views
Testing two potential interaction variables (or potential sources of effect heterogeneity) against each other
I have an experiment I have run, and I am testing for heterogeneous treatment effects (pre-registered and not fishing for any particular result!). Let's call the outcome $Y$ and the treatment variable ...
0
votes
1answer
49 views
Latin hypercube sampling with categorical variables
I would like to run sensitivity analyses on my agent-based model. My model has 20 parameters that I need to vary. I generated 8 artificial landscapes that vary in resource aggregation (r) and my model ...
0
votes
0answers
21 views
Experimental design for simulation experiment
I'm interested in investigating the effect of a treatment on survival times of a patient. The "patient" is a complex program with many interacting categorical and continuous variables. It also has a ...
1
vote
1answer
41 views
Which statistical test to use? Not really paired, not entirely unpaired
I am setting up an experiment where I would like to determine whether there exists a difference between the participants' average self reported mental effort in some condition A and condition B.
The ...
0
votes
0answers
17 views
quasi experimental study with replicated interrupted time-series design
I have a question about Quasi experimental study with replicated interrupted time-series design:
Can there be three different groups: two with different interventions and one with no intervention?
...
0
votes
1answer
20 views
Restricting block size in a factorial experiment
I have the following:
It has been decided to run an experiment involving 2
replicates of a $2^4$ factorial with factors coded A, B, C & D.
However, block size is restricted to 8
I'm not sure ...
1
vote
0answers
33 views
How should I analyse the results of my experiments with respect to their hyper-parameters?
I trained (and tested) a CNN with the same architecture on the same task (MNIST). However, I trained it several times: one for each combination of hyper-parameters. Each model is trained and tested ...
0
votes
0answers
28 views
1 experimental group - 6 control groups: ANOVA or t-tests or something else?
Objective: I want to find out, if a certain machine learning algorithm can select the right 100 stocks out of my stock universe of 2.000 stocks, such that they achieve better combined yearly returns ...
2
votes
1answer
50 views
Is it correct to run a mixed model (2-ways anova) on a unbalanced and incomplete design?
I am studying the effect of land use on biomass. I am struggling to find the right model to correctly analyse my data.
Data description
I have surveyed land use changes in 20 fields for 3 years. I ...
1
vote
1answer
43 views
Hypothesis Testing: Is a Tool Better Than Random?
I am testing a tool that tries to select the correct outcome. I am trying to do significance testing to see if the tool is better then choosing the outcome at random.
It picks from 4 categories, and ...
0
votes
0answers
25 views
Dot subscript summation notation used in design of experiments
Context
I couldn't find a clear explanation of this on this site, and
thought that it might be of use. I have provided part of the answer, which
anyone is welcome to build from. I'm not sure how to ...
