Questions tagged [experiment-design]
The study of how to structure an information-gathering exercise where variation is present.
1,556
questions
0
votes
0answers
3 views
How to choose the combination of factors that offer the best global response within a factorial design?
A factorial experiment was developed to find the influence of 3 factors in 2 response variables. The selected factors and their respective levels were:
The chemical used
Organic
Inorganic
The ...
1
vote
0answers
14 views
Designing an experiment to compare how multiple SMOTE variants affect multiple classification models on multiple datasets
For a university paper I want to test a hypothesis that one particular SMOTE variant outperforms two other SMOTE variants. By 'outperforms' I'm looking at using the F1 measure.
I want to test this ...
0
votes
0answers
7 views
Not sure how to start analyzing my dataset- could use any wisdom
I am new to statistics, and while I am not asking for anyone to do anything for me, I would appreciate and use any statistics wisdom that I could get. I was entrusted with some undergraduate research ...
0
votes
0answers
12 views
Adjustment for size of treatment difference
I routinely need to carry out tests for what may be best described as 'treatment recommendation protocols'. In the simplest case I have a model that 'predicts' optimal treatment based on some set of ...
0
votes
0answers
5 views
Appropriate ways of analyzing the influence of two independent variables on a reaction time?
I have the following experimental data. The dependent variable is a reaction time which has been measured on n = 30 test subjects. The reaction time was measured six times per test subject and the ...
1
vote
0answers
8 views
Power Calculations for Diff-in-Diff with Fixed Effects
I am conducting a field experiment. Where we observe two different optimization methods.
First 7 days pre-treatment period and 28 days treatment period.
To split treatment and control, we will have ...
0
votes
1answer
19 views
Time length for causal inference experiments
Let's say that I want to run a causal inference experiment, that is an experiment on historical data for an intervention that we were not able to perform a randomized controlled trial for. In the case ...
0
votes
0answers
14 views
Assignment rules in randomization experiment for causal inference
I'm learning the Casual Inference material on Coursera. I have a question about the following assignment rules in randomization experiment. The first page is for the letter notations. Briefly speaking,...
0
votes
1answer
28 views
Appropriate Statistical Test for Likert Data?
I am having trouble deciding the appropriate statistical test to use for a given experiment.
In the study, participants first recorded various feelings on a 4 point Likert scale.
Ex: How much do you ...
0
votes
0answers
13 views
Experiment Design: Best way to detect seasonality patterns in trading data?
I wanted to analyze crypto derivatives data and see if there is any seasonality/patterns to it. I have a daily dataset for about 400 days, and wanted to check for weekly/monthly/quarterly seasonality. ...
0
votes
0answers
7 views
Can I run a factorial analysis on secondary data?
I want to run a 2 by 2 independent groups factorial analysis of covariance on the dependent measure of X.
However, I have only access to the secondary dataset.
Would this cause any problem? Have you ...
0
votes
0answers
22 views
Statistical tests that can be run on differences between observations, including repeated measures?
I'm trying to figure out what stats I would have to use for my thesis.
For my first experiment, I am planning on getting roughness values from grasshoppers and rocks (from 4 sites, 5 grasshoppers and ...
0
votes
1answer
17 views
Defining test- and control groups for geo-test with balanced covariates and group size
I'm currently designing an A/B-Test on a geo level, where the test- and controlgroup will be made up of various (US) States.
Each state has a few specific characteristics (e.g. mean income, ...
0
votes
0answers
11 views
How to differentiate between 2 variable and 3 variable experiments, and experiments testing for interaction VS effect modification
In this article "Effect of age on muscle hypertrophy induced by resistance training" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3455754/pdf/11357_2002_Article_5.pdf),
I am trying to ...
1
vote
1answer
62 views
Clarification on the assumptions $E[u|x]=0$ and the $x_i$ being fixed in repeated samples in Wooldridge Introductory Econometrics
The author is writing on the assumption $E[u|x]=0$.
The part of the text which is not clear to me is this (the red lines emphasize where the critical portions are located) :
In the first piece I don'...
0
votes
0answers
19 views
What is the right approach to bucket users for algorithms with different coverage for A/B testing
I've couple of recommendation algorithms that I want to A/B test. Algorithm A has 90% user coverage and algorithm B has 95% user coverage. That means if the algorithms are asked to provide ...
1
vote
0answers
24 views
Compare data from two experiments within single logistic regression to determine effect of experimental design?
I've run two studies.
Study 1 (N = 112) examines the effects of two variables (A and B), each of which have two levels (a1, a2; b1, b2). For each variable, I'm interested in the effect those different ...
3
votes
0answers
30 views
Choosing training data inputs to optimize approximation
Suppose you have a smooth function $f^*:D_1 \times D_2\rightarrow\mathbb{R}$ that you observe with error as $f$ such that $$f(x,y)=f^*(x,y)+\epsilon$$ where $\epsilon$ has zero expectation (you can ...
0
votes
0answers
4 views
In Choce Experiments (DCE) creating cards from scenario sets must be random?
It will be difficult for me to describe it in one sentence, I will use an example. I would like to know if I can arrange the cards in the block combining scenarios other than randomly.
In Aizaki, H., &...
0
votes
0answers
8 views
Fractional, half factorial design (2^3−1 III , I = ABC) design: Sample size and within subject asignment
I need a little help planning a psychological experiment. We have three factors with two levels each. We are only interested in the main effects or rather simple effects (and possibly the moderating ...
0
votes
1answer
34 views
P-Values based on Experiment Data Aggregation
I'm trying to understand which interpretation of p-values for a linear regression model is correct given different levels of aggregation.
Let's say I am running a pricing experiment in a store-chain ...
2
votes
1answer
42 views
Does 'proper' experimental design, in terms of ensuring scientific validity, require that the same skilled individual perform all of the measurements?
I'm not sure if this is on-topic here. If not, then please tell me and I'll delete it.
I have a question about the proper way to design biomedical experiments that involve human-administered ...
1
vote
0answers
7 views
Between or within subjects?
So I have a 2x3 design with two categorical IVs, one is Country, the other is Message type (type A, type B, and control).
I've found published studies with very similar design that do both in between ...
0
votes
0answers
6 views
suggestion for multi-factor tests for an ordinal IVs and categorical DVs?
I have an ordinal DV and 3 categorical IVs. Two of the IVs have 2 levels, and one has 4 levels. I would like to do a hypothesis test to see if the mean of my DVs are equal, or if the variance between ...
0
votes
1answer
16 views
Control group participants
I have read a lot especially on RCT design but this hasnt been explicitly answered anywhere. Does the control group participants have to have 'opted-in' to the intervention and failed/not been ...
0
votes
0answers
19 views
Cross-sectional study design with no control group compares 2 treatment arms
There is a trend among medical research that assesses cross sectional (singular point in time) treatment comparisons between 2 treated groups in retrospective data. In other words the a person who ...
1
vote
0answers
14 views
Analysis of treatment effect with partially non-randomized groups
I have conducted an experiment in which participants were randomly assigned to six groups:
Control group
Control group with additional information about problem
Pre-set treatment group
Pre-set ...
1
vote
1answer
27 views
Treatment length in differences-in-differences
I have a question regarding how long the treatment has to be in a causal study? In my case I have a random shock that just occurred in one day for a treatment group (or just an event that exposed one ...
1
vote
0answers
18 views
Should ML control groups be random or drawn from low predicted probabilities?
If we wanted to build a propensity model which alloted a probability for some action eg purchase or cancel so that they may be preferentially contacted, and given 100 customers and a 50/50 split ...
1
vote
0answers
6 views
multi factor anova when only one factor is controllable and the others are not
Using an example of baking a crepe cake, assume that there are 3 layers to the cake, with each layer having 3 flavors. The bakery then assigns three bakers, one to each layer. The goal is to ...
0
votes
0answers
51 views
How do I test significance for this quasi-experimental test data?
Farm plot 1 (20 acres) produced this output over six years:
450, 460, 455, 445, 451, 462 (mean 453.83)
Farm plot 2 (20 acres) produced this output:
452, 458, 456, 443, 454, 459 (mean 453.67)
The ...
2
votes
0answers
26 views
Distribution of players in a computer game on different “levels”
Many computer games have different levels. Here is a description about levels on wikipedia, but I will explain more precisely what I mean below.
The idea is this: to optimize the experience with a ...
1
vote
0answers
29 views
Statistical power analysis in Python for Design Experiment problem
I'm diving into some design of experiment studies based on Design and Analysis of Experiments, 9th Edition by Montgomery. I've had some questions about the first time I faced power analysis in the ...
4
votes
1answer
43 views
A priori publication of statistical analysis plan (SAP) for clinical trials
This is somewhat of a regulatory question, rather than statistical question. But I assume that the readership of this community will be the most knowledgeable on this matter.
Question:
Is there any ...
1
vote
2answers
53 views
Calculating the degrees of freedom
Hi I wonder if there is a standard way to calculate the degrees of freedom. I am having a hard time understanding why these sums of squares has only one degree of freedom.
In one-way ANOVA (one ...
1
vote
0answers
16 views
How to evaluate the interaction between 3 factors with a single observation in each cell?
Some context: An experiment was conducted to evaluate the interaction between 3 factors in the removal of a contaminant from water. The selected factors were:
Coagulant: a chemical substance to ...
1
vote
2answers
32 views
When should one use a 1-Tailed or 2 Tailed test?
Let’s say there is a new product feature “X” that is going to replace the current product feature “Y”. The business wants to make a decision tto implement “X” only if it performs better than “Y”.
Here'...
0
votes
0answers
7 views
Binary DV across versions
I'm trying to figure out the correct statistical analysis to use for an unusual design.
The DV is a choice of one of 4 videos options. 2 of the videos are live and 2 are taped. I'm not interested in ...
0
votes
1answer
14 views
Logit First-stage and Logit Second-Stage in Experiment with 2 binary Instrumental Variables
We are heavily discussing how to analyze our experiment.
There is a binary treatment and a binary dependent variable. Therefore I think I should not use OLS but rather probit/logistic regression.
It ...
1
vote
0answers
56 views
Bayesian estimate of treatment effect for within-subjects and between-subjects design
Let's say I'm running an experiment with an experimental condition and a control condition, where four subjects in total provide two reaction times each. I'm interested in estimating the treatment ...
0
votes
0answers
36 views
Approaches for creating observation windows in a next-n-month regression model
I am interested in predicting whether someone will post at least once on social media in the next, say, 6 months (so, setting this up as a binary classification task). I am trying to determine the ...
0
votes
0answers
11 views
What are some plausible criteria for starting a statistical meta-analysis process?
Before stating the problem, let me indicate my background so the community can understand my capabilities, or lack thereof. I am a trained biologist. However, during my bachelor's I studied many CS ...
1
vote
0answers
23 views
Question on how to do the randomized paired design on LED bulbs
Want to compare the average lifetime of LED bulbs under two different temperature settings, namely, $S$ degree Celsius and $T$ degree Celsius. Suppose that the distribution of the bulb’s lifetime is ...
2
votes
1answer
34 views
Is it acceptable to remove a block of data from an experiment due to outliers
I performed an analysis a while ago, and the experiment was blocked for production day (3 different production days). Two of the production days had terrible outliers, and after looking at the Cook's ...
0
votes
0answers
16 views
How to make phrases in experimental conditions as similar as possible?
In behavioral economics and psychology, there are many experiments where participants receive variations of a piece of text depending on the experimental condition they are assigned to. For example, ...
0
votes
1answer
37 views
Making sense of an actual equation obtained after Box-Cox transformation
I have recently performed and analysed an experiment and I am currently stuck on making sense of the outcome. Any help would be much appreciated.
The experiment consists of a simple DC voltage source ...
0
votes
0answers
10 views
Multi Arm Bandits - Purchase Amount Zero Inflated
All the examples I have seen for multi-arm bandits (MAB) applied to say online advertising are for cases where the reward is binary (e.g. click, no click) or continuous (e.g. assuming a normal ...
1
vote
0answers
17 views
D-Optimality for regression of polynomial models in one variable with missing terms
Let's say I have a model that looks as follows:
y = x + ax^3 + bx^5 + cx^7 + dx^9
Given n free choices for x as input measurements how can I determine which x's I should input to best determine a,b,c,...
1
vote
1answer
41 views
Sampling plan for surrogate modelling
I am trying to develop a surrogate model for a thermoacoustic engine. The engine is modelled in a program called DeltaEC. I have developed automation code so that I can change parameters, run the ...
0
votes
0answers
19 views
Standard deviation increase for one-way ANOVA
I am reading Montgomery's Design and Analysis of Experiments (Wiley, 2012), and on page 108, it is written
If the treatment means do not differ, the standard deviation of an observation chosen at ...
