Questions tagged [experiment-design]
The design of experiments (DOE) is the study of how to structure an information-gathering exercise where variation is present. In statistics, controlled experiments are usually implied, meaning that study units are randomly assigned to conditions where levels of a variable have been independently manipulated (prototypically into treatment & control groups).
0
votes
0answers
7 views
Two way blocking experimental design
I'm doing experiment where I'm checking how are subjects learning doing some task (5 consecutive rounds) with or without help. Some subjects will have help visible all the time, others won't.
I'm ...
1
vote
1answer
23 views
Which statistical test to use? Not really paired, not entirely unpaired
I am setting up an experiment where I would like to determine whether there exists a difference between the participants' average self reported mental effort in some condition A and condition B.
The ...
0
votes
0answers
7 views
quasi experimental study with replicated interrupted time-series design
I have a question about Quasi experimental study with replicated interrupted time-series design:
Can there be three different groups: two with different interventions and one with no intervention?
...
0
votes
1answer
18 views
Restricting block size in a factorial experiment
I have the following:
It has been decided to run an experiment involving 2
replicates of a $2^4$ factorial with factors coded A, B, C & D.
However, block size is restricted to 8
I'm not sure ...
1
vote
0answers
33 views
How should I analyse the results of my experiments with respect to their hyper-parameters?
I trained (and tested) a CNN with the same architecture on the same task (MNIST). However, I trained it several times: one for each combination of hyper-parameters. Each model is trained and tested ...
0
votes
0answers
27 views
1 experimental group - 6 control groups: ANOVA or t-tests or something else?
Objective: I want to find out, if a certain machine learning algorithm can select the right 100 stocks out of my stock universe of 2.000 stocks, such that they achieve better combined yearly returns ...
2
votes
1answer
39 views
Is it correct to run a mixed model (2-ways anova) on a unbalanced and incomplete design?
I am studying the effect of land use on biomass. I am struggling to find the right model to correctly analyse my data.
Data description
I have surveyed land use changes in 20 fields for 3 years. I ...
1
vote
1answer
40 views
Hypothesis Testing: Is a Tool Better Than Random?
I am testing a tool that tries to select the correct outcome. I am trying to do significance testing to see if the tool is better then choosing the outcome at random.
It picks from 4 categories, and ...
0
votes
0answers
19 views
Dot subscript summation notation used in design of experiments
Context
I couldn't find a clear explanation of this on this site, and
thought that it might be of use. I have provided part of the answer, which
anyone is welcome to build from. I'm not sure how to ...
0
votes
0answers
16 views
What test to use with many groups and interactions
I am trying to design an experiment with impact of many factors on one variable, the basic idea is to measure variable A when facing factor 1, factor 2, factor 3, etc.
Secondly I would like to ...
0
votes
0answers
32 views
How to statistically compare feature sets with different levels of data collapsing over category and training splits?
Background: As part of my research into image classification from small numbers of examples:
I created 4 disjoint feature sets by extracting activations from different layers of a deep neural network ...
1
vote
1answer
30 views
What is the Level of Evidence of “simultaneous self-controlled study”?
I guess this is an undefined thing, and would like to know your opinion. What is the Level of Evidence (as defined in the link by Oxford https://www.cebm.net/2009/06/oxford-centre-evidence-based-...
0
votes
0answers
7 views
How to determine the sample size for a control group when conversion is low
I have to test a binary classification model, and I'm not sure how to define the control set.
I have a 300k potential clients, and I have to choose the most probable 10k that will buy my product.
The ...
0
votes
0answers
13 views
Control learning effect in experimental study
I conducted an experiment where participants drove in a driving simulator.
They performed practice runs, then a baseline and then 4 different treatment runs.
For each of the runs (baseline & ...
0
votes
0answers
20 views
Degrees of Freedom in 2k factorial design
I am working on a 2k factorial design experiment, and got stuck at one point. If the suppose there are only 2 factors A and B (K=2) each with only one replicate each.
This means n=2.
Then according ...
1
vote
1answer
28 views
Performing repeated measures ANOVA on unstacked data in R
I am trying to do a repeated measures ANOVA on unstacked data for my experiment. For background, my experimental design is as follows: I have 160 stimuli divided evenly into 8 counterbalanced list, ...
1
vote
0answers
15 views
A/B testing use case and test set up
Suppose I want to know whether a change to a website (such as changing the way the homepage looks) has no effect on a metric I care about, such as conversion. Can a frequentist (not Bayesian) A/B test ...
0
votes
0answers
24 views
Finding the expected value of sum of squares
I am still confused about how to get the expected value of sum of squares from a statistical model.
For example,
for the model: $y=\tau_i+\beta_{j(i)}+ (\tau\beta)_{ij}+\gamma_{k(j)}+(\tau\gamma)_{...
0
votes
0answers
6 views
Sample size using G power repeat experiment
We are trying to use G power to calculate the sample size in an experiment that will be repeated on two separate days.
We will have 6 groups and will repeat the exact same study with another 6 groups ...
0
votes
0answers
13 views
What study design should be used for a prevalence study and risk factor association
I want to estimate the prevalence of Hypertension in Diabetic patients and to check whether the prevalence is different in smoking and non-smoking people. Can someone help how to design this study ? ...
0
votes
0answers
15 views
Standard Text on Experimental Design [duplicate]
I am interested in learning experimental design largely from a theoretical/probabilistic standpoint. I have searched the internet/moocs on this topic and largely it has been R/SAS driven rather than ...
0
votes
0answers
29 views
Multiple regression data analysis
https://docs.google.com/spreadsheets/d/1czrXmJuxvwalDwNDLHIvt64QqNb8cSTahEEK8DIiQPM/edit?usp=sharing
the study: "right and left visual hemifields and duration of the presented stimulus were ...
0
votes
1answer
65 views
Correct approach to comparing sub-group means across multiple treatment conditions
Let's say I have an experiment where I randomly pair people up with either a male or a female player, and they play a game together. I then measure some dependent variable -- call it ...
1
vote
1answer
33 views
How to perform augemented DOE in R [closed]
I have run a series of experiments for a continuous relationship between a response (y) and a single covariate (x). The model for this relationship is probably somewhat logarithmic, so y = Beta log(x)....
1
vote
1answer
28 views
In propensity score matching, what violations or implications may result from having fitted propensity scores that are not centered at 0.5?
I currently have a procedure doing propensity score matching, and I use the fitted propensity scores (obtained via a glm call) and match on those. It turns out that I have about 60-70% more fitted ...
6
votes
2answers
135 views
Who is right, the statistician or the surgeon?
Consider the case described below, from Peacock (1972). This passage seems to imply the young statistician is making a smart, correct statement.
But is he?
0
votes
0answers
13 views
How to Analyze A Confounded Design in R?
I have a $2^5$ factorial experiment, with one replicate, that is divided into $4$ blocks such that treatments $ACDE$ and $BCD$ (and $ABE$ as a result) are confounded with the blocks. I understand what ...
0
votes
0answers
25 views
Statistical inference of a variable based on categorical respone levels.
Suppose that we have a variable $X$ with three levels $1$ ,$2$ and $3$ accordingly.Also we have a catergorical respone variable with two levels $Y=\left\{Healed,Not Healed\right\}$ which tells as if ...
1
vote
0answers
34 views
What regression model to use?
I am trying to conduct a study on how public transport use changes with the introduction of a road pricing policy, what regression model should I be using?
1) A simple Log-linear model
2) A ...
-1
votes
0answers
12 views
What statistical test would I use to compare the effect of pre-existing attitudes on post intervention scores?
Basic design:
Asking people to indicate their attitude toward getting a flu shot (4 options: very negative, mostly negative, mostly positive, very positive)
Then half of the people will receive ...
0
votes
0answers
17 views
Test to check reliability of sample splitting procedure
As someone with little background in statistics, I am trying to test the reliability of the splitting procedure for screen analysis currently done in the lab (if the particle size distribution will be ...
0
votes
0answers
20 views
post-hoc, pairwise PERMANOVA for factor with three levels
Goodmorning,
I have a mixed, multifactorial design with 4 factors. I've run my PERMANOVA which returned a significant result for fixed factor: year (3 levels). My thesis pertains to changes over time,...
6
votes
3answers
230 views
Regression and causality in econometrics
In regression in general and in linear regression in particular causal interpretation about parameters is sometimes permitted. At least in econometrics literature, but not only, when causal ...
0
votes
0answers
13 views
Correct group of factorial and repeated-measure ANOVA
I'm currently having trouble defining what type of ANOVA I need to run as well as how to run it correctly. This is the following setup:
I have 4 groups of 30 people doing a 2-session task (one after ...
0
votes
1answer
33 views
Algorithmic or structural limitations of space-filling Latin hypercube sampling
I'm new to Latin hypercube sampling, and am trying to understand if the somewhat odd sampling that results from the Matlab function lhsdesign is a limitation of the ...
1
vote
1answer
39 views
Why can you not bypass the strong ignorability/unconfoundness assumption via iterated expectations?
Suppose we have that $\left(Y(1), Y(0)\right)$ are potential outcomes with $X$ being the covariate and $Z$ the treatment assignment. Typically in causal inference, one will assume strong ignorability ...
2
votes
1answer
144 views
non stochastic regressors
In the multiple linear regression analysis if regressors are non-stochastic the causal interpretation of parameters is automatically permitted?
I think so, because it seems me that the model can be ...
0
votes
0answers
30 views
When should we include the “Replicate” in an ANOVA table for Factorial Design?
When I have been learning how to do Factorial Designs ($2^k$ mainly) the ANOVA table would usually include the main effects, all the interaction effects if necessary, and the error. However, on one of ...
1
vote
0answers
43 views
Is this a valid experimental design?
An experiment to test 2 new treatments was performed in the following setup:
Animals were divided into 2 groups: young (A) and old (B). Animals were present for 3 periods. Each group consisted of 81 ...
1
vote
1answer
39 views
How to use Design of Experiment (DoE) to reduce the number of simulations?
I am planning to do simulation for parametric study and there are 9 parameters in total. I was suggested to use DoE to reduce the number of simulations that I need to do. I studied the basic of DoE ...
0
votes
0answers
8 views
Randomised controlled block experiment - choosing the blocks
I'm confused about how to decide blocking, and what that means for experimenting with it. I've seen other questions on the site and don't feel that they answer this, I think this is more basic.
For ...
0
votes
1answer
29 views
Experimental Analysis with Several Discrete Treatments
I am analyzing data that are originating from an randomized experiment and I am new to this. There was one control groups and three different treatment groups. The treatment groups are discrete and ...
0
votes
1answer
36 views
Difference between Multistage Sampling and Stratified Random Sampling?
I know the question is a very elementary one, but I simply cannot understand the difference other than the fact that an SRS is a form of Multi-Stage Sampling. Can anyone provide a simple example(s) to ...
0
votes
1answer
62 views
non stochastic regressors and causation
Randomized controlled experiment is base case for causality (also) in regression.
However currently I’m analyzing the role of causality in linear regression as shown in many econometrics textbook. ...
3
votes
1answer
71 views
Proof that random assignment guarantees the same probability for the individual to be selected?
Consider a completely randomized design, where every unit is randomly assigned to a treatment group. Let's say we have 30 observations and 3 treatment groups. When we choose one observation at random, ...
2
votes
1answer
41 views
When should a randomised Latin Square be used rather than a standard Latin Square
I'm looking at Latin Squares. I've seen standard Latin squares, and Latin squares when the rows are randomised, then columns are randomised.
When would one be used over the other?
It seems as ...
2
votes
0answers
31 views
How does the “BiasAdjust” term in Match() function from Matching R package work?
In the Match() function in the Matching R package, it takes in inputs like:
...
1
vote
1answer
31 views
For the Match() function in the R package, “Matching”, what algorithm is used for propensity score matching?
In the Matching package in R, one can conduct propensity score matching if propensity scores are passed to the ...
2
votes
2answers
35 views
In observational studies with propensity score matching, why is the ATT usually reported? Is the ATE usually not available?
In most propensity score literature on observational studies, the ATT is usually reported and sought after. However, the ATE is usually not reported and in some cases I've read papers that claim you ...
1
vote
1answer
34 views
What is the difference between cross-over and repeated-measures designs?
I know that a cross-over experiment (of any number of treatments/levels) necessarily has a repeated-measures design, but that not all repeated-measures designs are cross-overs. However, I wasn't able ...
