Questions tagged [experiment-design]
The study of how to structure an information-gathering exercise where variation is present.
1,505
questions
0
votes
0answers
12 views
experimental design in R project
I want to know of any repositories that contain complete experimental design in R covering basic tests and analyses?
I want to take a top-bottom approach to learn step by step through a real project ...
1
vote
0answers
34 views
Correcting linear regression for change in air temperature over time
Say I collect temperature data over 5 square kilometers over the course of an hour in the evening but I want to analyze the data as though they were collected all at the same time. Naturally the area ...
2
votes
1answer
27 views
ANOVA and independent samples
Let's say there is an experiment of determining the best cookie recipe with 4 categorical factors. There are $S$ subjects that eat and rate cookies made using $R$ different recipes. There are $R$ ...
1
vote
1answer
11 views
Regarding the Plackett-Burman screening design
I am starting to implement screening designs in my optimization work. The one I am most familiar with is the Plackett-Burman design, so I am starting there. I have written my own scripts in MATLAB but ...
0
votes
1answer
21 views
Covariate for consistent estimation of treatment effect
I come across this sentence as I was reading journals: "to consistently estimate the treatment effect, any covariate X introduced must have equal expectations in treatment and control: E[X|Di= 1] =...
0
votes
0answers
5 views
Explanatory variables (exposures) that can be thought of as additive - best practices over multiple models?
I am modeling an exposure against an outcome (not COVID-19 related). The intended purpose of my analysis is to better understand (from a descriptive perspective) the nature of this exposure in regard ...
1
vote
1answer
7 views
Quasi-random sequence with discrete variables with differing number of levels
My question is probably worded incorrectly but here it is:
I have (say) 3 discrete random variables:
x1: has 15 levels (uniform pdf for simplicity)
x2: has 3 levels (uniforms)
x3: has 4 levels (...
1
vote
0answers
23 views
How many points for statistical significance in design of experiments
I am running an experiment that includes 27 input data points. The function yields results if we input $2 \leq x \leq 27$ points.
I divide my data points in 9 sets of 3 points each.
Now, I want to ...
0
votes
0answers
5 views
Should I pool samples or leave them separate before running a PERMANOVA?
I have a dataset of fish species from different sites within a harbour collected over 17 years. The dataset consists of 1,042 sampling transects collected at 6 specific locations where fish were ...
1
vote
0answers
29 views
How to deal with large combinations of factors
I am faced with a new problem that I cannot really tackle with the way I usually do my analyses and would appreciate any input you may have.
The research goal is to identify the factors and ...
0
votes
0answers
16 views
analyze / model the difference between experiment and control group
In an experiment with experiment vs control groups, how to train model to explain the different results between experiment and control group (ie, has treatment vs no treatment)?
Eg, customer ...
5
votes
1answer
79 views
+50
Identifying the correlation between a slope and a level
Throughout this post, I assume at least second moments exist. Consider a heterogeneous linear treatment effect model of the form:
$$Y_i = \alpha_i + \beta_i X_i$$
where $\alpha_i, \beta_i$ are ...
0
votes
0answers
11 views
Randomised or generalised block design?
sorry to ask a basic question, but I have not been super familiar with those types of experimental design yet and took over an already established experiment.
Imagine those two set-ups:
Three ...
1
vote
1answer
21 views
Ordinary least squares estimation explained in “Optimal Design of Experiments” by Goos/Jones, questions about some equationsEq
I am currently trying to become familiar with design of experiments with the book “Optimal design of experiments” by Goos and Jones. In chapter 2, they discuss the use of a Plackett–Burman type design ...
0
votes
0answers
14 views
How can I simulate an experiment and account for trial- and subject-level statistics?
I'd like to write some code to simulate a typical experiment but I have a bit of an issue and could use some feedback/ideas. My plan was to simulate a number of individual trials (each with a binary ...
0
votes
0answers
4 views
How can I evaluate the affect of risk scores on net revenue for an a/b test when there's some interaction from another test?
We conducted 2 separate A/B tests: the first has 3 variants - control, treat_a, treat_b where we modified a particular risk score in the hopes that we would increase net revenue. Coincidentally, there'...
1
vote
0answers
11 views
Which test should I use for a three alternative forced choice experiment?
I am running an online survey experiment where participants are randomly assigned to different treatment groups. They are asked to make a three alternative forced choice (3AFC) decision, i.e., pick A, ...
2
votes
1answer
31 views
Determine the variabilty of the volume given the length, width and height
I am to solve a problem where I am given the mean value, $\mu_i$ and variance, $\sigma^2_i$ where $i\in{L,W,H}$ which represents the Length, Widht and Height related variables respectively.
The ...
1
vote
0answers
22 views
Power, split plot designs, and mixed models
My teammate and I are trying to determine the best way to allocated limited resources (shade cloth and fencing) in an experiment. We are considering 2 factors: light availability (2 levels) and water ...
1
vote
1answer
33 views
How to Test Significance of Trading Strategy? [closed]
I am wanting to try out a new trading strategy, and it has proven successful on a simulator for the past few days. I do want to collect some more data and see if my results are statistically ...
0
votes
0answers
34 views
gender effect measurement
An interesting study of gender effects in chess showed that when women played men without knowing that they were playing men, they performed better than when knowing they were playing men.
I was ...
0
votes
0answers
4 views
Is a GLMM applicable in a non-full factorial experimental design with two dependent fixed effects?
At the moment I plan the statistics for an upcoming experiment in Biology with two fixed effects whose effect on a fitness parameter of a parasite I would like to test. The two fixed effects are the ...
1
vote
1answer
26 views
Insufficient power from pwr.p.test
Using the pwr package, I determined the optimal sample size of 38. However, when I ran simulations using this sample size, I only get a power of 70%. What's going ...
0
votes
0answers
19 views
strange ordinal regression - does it exist, what is it called, how do I do it properly
My daughter is making cookies for her science project. She is trying to "use statistics to optimize the recipe". She has asked for my help in the science part.
Currently:
She wants the ...
0
votes
0answers
9 views
Question about Within Subjects V.S. Between Subjects
What are two advantages and two disadvantages of within-subjects designs (designs with repeated measures) in comparison with between-subjects designs?
2
votes
0answers
22 views
Experimental Design for count data in a (4X2) Factorial with replication
I am in need of help choosing a statistical model for count data with experimental replication.
The Experiment
(4 X 2 Factorial experimental design)
-Factor A = Control, Low, Medium, High.
-Factor B = ...
1
vote
0answers
8 views
Sample Inclusion Criteria for an Intent-To-Treat Analysis in an Encouragement Design With One-Sided Non-Compliance
This question is about how to define the sample in an Intent-To-Treat analysis of an experiment with balanced pre-assignment and an encouragement design. This is a bit different than the usual setting ...
0
votes
0answers
29 views
Split-Split-Split Plot Design
I have an experiment which was set up as a split-split-split plot design and repeated the experiment a second year.
2yrs x 4 reps x 2 levels of Factor A X 5 Levels, of factor B X 2 Levels, of Factor C ...
0
votes
1answer
36 views
Analyzing a counterfactual in observational studies?
In observational studies, when randomization is not an option (everyone is treated), it is necessary to estimate a counterfactual of what would have happened in the absence of treatment.
Once you ...
2
votes
1answer
26 views
How do you analyze an unbalanced fractional factorial design?
I'm interested in learning about unbalanced fractional factorial designs. If they are possible to analyze, what are the drawbacks to using them? Just usual loss of orthogonality? How does analyzing ...
1
vote
0answers
27 views
Confidence interval of Relative Change using Delta method
I have the following equation for relative change (RC):
RC = (P1 - P2)/P2 * 100
Here, P1 and P2 are both ratios. For example, P1 = A1/n1 and P2 = A2/n2, where n1 and n2 are the sample sizes of both P1 ...
0
votes
1answer
23 views
Remove points, but preserve diversity
Given a finite set of points in $\mathbb{R}^n$, all at finite distances from each other I would like to remove a certain proportion of points, in such a way as to penalise clustering. For example for $...
0
votes
0answers
5 views
How should I test for the effect of an interaction with “uneven” levels?
Basic Effect
Is there a difference between the conditions of "Factor A"?
That is, A1 vs. A21.
In A1 participants see a paragraph of text.
A2 is a combination of A1 + another paragraph of ...
3
votes
1answer
45 views
Questions about Data Analysis
For my research I'm examining whether applying a certain promotional intervention improves the amount of misshapen products bought. The study contains of two parts. In part 1 Scanner data from ...
0
votes
1answer
24 views
Question regarding Optimal Designs of experiments
I'm a bit unclear on the concept of optimal design of a data matrix $X$. I propose a small example to work through:
Suppose $\epsilon_i \sim N(0, \sigma^2)$ are i.i.d., and I have some experiment ...
0
votes
0answers
6 views
Competition with randomized teams that eventually teases out best individual performers
I seek to set-up a competitive league among friends. ~8 individuals total. ~50 total 2-on-2 matches (2 matches played simultaneously per round). Pairings shuffled after each round. Goal: individual ...
1
vote
0answers
42 views
D-optimal design with nuisance parameters
I am a mechanical engineer trying to develope an optimal design of experiments in a problem with nuisance parameters. I would like to calculate the parameters $\mathbf{d}$ to optimally estimate ...
0
votes
0answers
5 views
Number of Tests necessary for object detection (test/experiment design)
I want to find out the number of tests necessary to validate certain performance characteristics of an object detection system with a certain confidence. The system just detects if a relevant object ...
1
vote
1answer
28 views
Testing for differences between pooled biological samples
I am trying to test for differences in absorbance between a control and treatment group. Due to cost, each measurement is derived from the mixture of 5 independent samples. Overall there are 5 ...
1
vote
0answers
25 views
How to estimate this effect?
I have data with columns like this:
[individual, song, is_after_drinking (IAD), enjoyment]
I have observations on an indivudal J rating a song K (n=800 ...
3
votes
0answers
31 views
Statistically, is it effective to design exams to get harder as you answer correctly?
I think we have all taken exams where as you answer more questions correctly, the exam gets harder. Intuitively, this is obvious why...If an IQ test had questions for 3rd graders, everyone would get ...
1
vote
1answer
25 views
Treating site and year as an integrated factor
I’m analyzing an experiment where we collected data across 4 different treatments. These treatments were replicated over two years.
In each year, we set-up the experiment at 2 different sites.
In ...
0
votes
0answers
10 views
How to test if term is correlated to news category
Suppose we have a corpus of news articles that contain a term. We also have the category for each article, which ranges from Sports to Politics, etc. What statistical approach would we take to ...
1
vote
1answer
29 views
A difference in difference experiment with multiple interventions
Say you have a control group and an experimental group and you have verified the parallel trends assumption for them. Now say that at the time of intervention, the control group gets 9 separate ...
4
votes
2answers
351 views
Is peeking in A/B tests bad even if you don't stop your experiment?
I have been asked a test about A/B testing that I'm unsure how to answer.
Say you're running a conversion rate experiment, and my A variation has a conversion rate of 2% and my B variation has a 2.2% ...
5
votes
1answer
41 views
A theoretical question on fractional factorials
Very much in need of help interpreting this if anyone is available.. Thank you!
Fractional factorials that include many factors can sometimes have a surprisingly low number of experimental conditions ...
4
votes
1answer
31 views
Analyzing clustered (repeated measures) data with very few clusters
What is the most appropriate method for analyzing clustered data (i.e., repeated measurements nested within subjects) when there are very few clusters (e.g., 3 subjects)?
Here's a brief description of ...
3
votes
1answer
30 views
How to build a GLMM that observes the years since experimental design was established?
Hello, my first question, quite individual, so I find it difficult to relate already answered questions to mine.
I have observed the vegetation development in forests of 5 different areas (...
7
votes
0answers
59 views
How to best respond to investigator who wants to study secondary outcome
I'm co-investigator on a clinical trial where the we are studying the effect of an intervention. The study is powered to detect a clinically meaningful change for some primary endpoint. My questions ...
0
votes
0answers
43 views
Appropriate statistical test to use
I am running an experiment where I am testing the difference in effectiveness of constructed wetlands. Essentially, I have a control with unplanted wetlands and treatment with planted wetlands and its ...
