Questions tagged [experiment-design]
The study of how to structure an information-gathering exercise where variation is present.
1,385
questions
1
vote
1answer
10 views
Which additional experiments would best improve the model?
I have been given some data on a biotechnological process stemming from a multitude of different (kind of independent) experiments. The process has a lot of potentially relevant inputs (temperature, ...
0
votes
0answers
19 views
experimental design and linear regression
In an experimental design, randomly assigned participants into treatment group and control group and there is no statistical difference(using t-test) between two groups. The treatment group was ...
0
votes
0answers
12 views
Sample size calculation for AB testing in offline scenario
I have sales data for 500 stores and lets say I want to run an AB test on a sample of test and control stores with the objective of getting statistically significant increase in sales(lift). My ...
0
votes
0answers
4 views
How to proceed when the data created from using a fractional factional design gives unsatisfying models?
We've got 10 factors to investigate. Some are binary, some are continuous. Initially we created a 2-level Plackett-Burman design (using min and max for the continuous factors) with 16 experiments (...
0
votes
0answers
13 views
List Experiment “ictreg.joint” error: “log-likelihood is not monotonically increasing”
I'm trying to use "ictreg.joint" function in "list" package in R by Imai and Blair to use the predicted responses from the list experiment as the explanatory variable to analyze an outcome variable. ...
0
votes
0answers
5 views
Interaction Effects, Zero or non-Zero in 2 factor factorial design?
In a 2 factor factorial design it is common to use the following linear model and the associated notation:
$y_{ijk}=\mu+\alpha_{i}+\beta_{j}+(\alpha\beta)_{ij}+\epsilon_{ijk}$
$(\alpha\beta)_{ij}$ ...
0
votes
1answer
13 views
MDE minimal detectable effect
I m not able to figure out how to calculate MDE(minimal detectable effect) when designing an experiment. I see that there are calculators to do it for you like this one: https://bit.ly/2XgMjld
For:
...
0
votes
0answers
18 views
detecting correlation between and significance of non-linear model parameters
I have a set of observed data y,obs that I am modelling using a non-linear model (currently in EXCEL though I also have a Python model). There are 4 input ...
4
votes
2answers
60 views
How to include an interaction with sex
I am performing linear regression on a dataset and want to include an interaction between sex and another covariate. However, in my model sex is coded as $0$ for female and $1$ for male.
If I have ...
0
votes
0answers
10 views
How can I apply a design effect of 2 on my fisher's test and/or regression?
I'm trying to use R or statistical analysis on a multicluster sampling and specifically to inform it that I have a design effect of 2.
I've familiarised myself with the survey package and tried to ...
0
votes
0answers
16 views
How to deciding on the number of experimental runs in R for orthogonal fractional factorial design?
Presently I am working on designing a questionnaire for my discrete choice experiment. I want to generate an orthogonal fractional factorial design for the following problem-
The respondent has to ...
1
vote
0answers
32 views
Experiment design advice
Iād like to understand a good experiment design for the following example:
Say sales people are trying to find the ideal time (in days) to send a follow up phone call to increase conversion (a sale).
...
0
votes
0answers
18 views
How to Deploy Contextual Bandits in Online Experimentation Platform?
This question is about how to deploy contextual bandits(CMAB) in the context of web site optimization and online experimentation. I implemented contextual free MAB(MAB). When I run a MAB experiment, I ...
1
vote
0answers
10 views
Factor analysis with missing data
I am looking for methods that allow me to do an exploratory factor analysis with missing data. The reason I need to account for missing data is that we will try to collect a large number of variables ...
1
vote
1answer
21 views
Generating a full rank model matrix in R
For a particular tool I need a model matrix which allows me to build desired contrasts and which is a full rank, i.e. no columns are linear combinations of other columns.
The experimental design ...
1
vote
1answer
23 views
How to interpret coefficients of a cross-lagged panel analysis
I'm trying to understand the coefficients of this cross-lagged panel analysis.
Are they correlation coefficients? are they regression coefficients?
I'm not sure how can I interpret them if I have no ...
1
vote
0answers
22 views
Compare performance of libraries - test selection
I am trying to perform a comparison between three implementations of the same algorithm but using different libraries. However, I am not sure about my methodology. I am measuring elapsed time as a one ...
1
vote
0answers
8 views
Aggregate effects of randomized experiment
Consider the following two-equation model for observational data from a randomized experiment:
(1) $y_{ij} = \alpha + \beta D_{ij} + \gamma X_{j} + \varepsilon_{ij}$
(2) $D_{ij} = \gamma + \delta ...
0
votes
0answers
17 views
$\mathbb{G}=(1,B,T,B \times T)$ is an orthogonal design
I have an orthogonal system with 2 factors, B and T.
I know that B and T are orthogonal geometric. But I also need to show that that the system
$\mathbb{G}=(1,B,T,B \times T)$ is an orthogonal ...
0
votes
0answers
19 views
Randomized Block Design - Feature is Now Insignificant
I'm working with COVID-19 data for a class project. Our initial randomized design revealed that the factor Income played an insignificant role in predicting the response variable Mortality Rate (p > 0....
1
vote
0answers
26 views
help requested for mirror polishing data set
In space satellites, mirrors are used to guide incoming light to detectors measuring the features of interest. It is crucial that these mirrors are very smooth so that they do not unduly scatter the ...
1
vote
0answers
40 views
What is the dependent variable in an Item Count Technique Regression (List experiment)
I try to understand the logic behind an Item Count Technique Regression (Imai 2011; Blair & Imai 2012). I am especially interested in the ML estimator for the multivariate regression analysis (...
0
votes
0answers
36 views
Optimal point pairs for estimating the distance function in a metric space
Iām looking for the name of the methods or subfield that deals with problems like this:
Assume that we have $n \times m$ points over a regular grid.They are embedded into a metric space. We can get ...
1
vote
0answers
14 views
What type of observational study only has one experimental group and one control group?
I'm looking at studies which have one experimental group (people with tinnitus) and one control group (an age and sex matched normal population).
The studies take the two populations, check their ...
3
votes
1answer
36 views
Am I violating any causal inference assumptions?
Consider a large-scale AB test with two variants: control and treatment. Assume we want to compare the two variants with respect to the following metric: "Proportion of users who streams more than X ...
1
vote
0answers
13 views
How are terms in SS (corrected sum of squares) independent?
What does the highlighted test mean? How are the terms not independent? Can someone elaborate in detail?
1
vote
0answers
27 views
Matched Pairs design definition
A horticulturist is planning a greenhouse experiment with 20 individually potted tomato plants to compare two fertilizer treatments. Ten plants are randomly assigned to fertilizer A, and the remaining ...
1
vote
0answers
49 views
Is there value in assigning a dimension-specific weighting when calculating the main effect?
I've been looking into factorial designs and noticed an interesting property. Is this simply a mathematical trick or can we attribute some 'meaning' to it?
Say we have a $2^k$ design where k = 3 here....
0
votes
0answers
3 views
Random assignment of treatments to half of a group [migrated]
I am in the process in randomly assigning treatments for an experiment. I have four sites (Site1, ... Site4) where 12 experimental units (e.g., 1 ... 12) are replicated four times (e.g., 1 ... 4). For ...
2
votes
0answers
9 views
In A/B test, how to deal inconsistency between unit of experiment and unit of analysis in the experiment design?
In the experiment, the metrics we care about is clicks per page (#clicks / #pages).
When performing the experiment, the unit of diversion we randomized on is "user" instead of "page".
I'm confused ...
0
votes
0answers
12 views
How to measure the repeatability in an experiment design?
I'm performing a design of experiment (CCD) with
6 factors: H2, CO, C2H2, ...
4
votes
3answers
46 views
What does one mean by a 'random variable' in the real world?
I want to understand what it means for process to be random or for an object to have random properties.
Contrary to what I used to believe and to popular belief J Schmidhuber argues that true ...
1
vote
0answers
25 views
Why is the Tukey method not appropriate for covariance analysis when estimating treatment effects?
In the book 'Applied Linear Statistical Models' chapter 22 it states that the Tukey method is not appropriate for covariance analysis. The context is in estimation of treatment effects. Instead, when ...
48
votes
8answers
7k views
Statistical tests when sample size is 1
I'm a high school math teacher who is a bit stumped. A Biology student came to me with his experiment wanting to know what kind of statistical analysis he can do with his data (yes, he should have ...
1
vote
0answers
43 views
Randomised Block Design
Suppose the response of three different treatments, $A$, $B$ and $C$ are measured in two different hospitals of a country. The data are given below.
My question is: so far I understand it is a ...
1
vote
1answer
22 views
What does 'control' in controlled experiment refer to?
Does 'control' in controlled experiment refer to:
Having a control group for comparison, i.e
"an experiment or trial that uses controls, usually separating the subjects into one or more control ...
0
votes
0answers
48 views
Effect size in design of experiment
I set up an experiment design (CCD) with
6 factors: H2, CO, C2H2, ...
0
votes
0answers
28 views
A/B testing use case
I am learning A/B testing and so far I have learned the theory and am trying to solve some use cases. I am working on a website data which has total visits and clicks at day x country x test/control ...
0
votes
0answers
37 views
Sample size in A/B testing and its impact on test being positive
How do I choose the right sample size for A/B test? looks like if I increase the sample size the test can always result in positive
Do we use t-test if my sample size is more than 30? because in t-...
0
votes
0answers
13 views
Estimability of Pararmeteric Functions and Elementary Contrasts
Why do we say that a linear parametric function is estimable when it is in form of an elementary contrasts?
I understand that a parametric function is estimable when E(l'Y)= c'Ī² where l=(l1,l2...ln)' ...
0
votes
0answers
23 views
Statistical approach for when respondents choose treatment
Overview
I'm designing a study that maps onto a real-world process where people use multiple pieces of information to make a decision. I specifically want to know (1) what pieces of information do ...
0
votes
0answers
23 views
Experimental Design
Given a coach wants to study the effect of practice techniques on the rate at which his students are able to learn new skills in soccer. He chooses 4 different drills (shooting, passing, dribbling, ...
1
vote
0answers
27 views
Do I have a true within-subject design?
I am currently in the design phase for an experiment on different packaging attributes and their effects on an actual purchasing choice. In this context, I am asking myself a possibly trivial question:...
5
votes
2answers
157 views
Asking nature a single question or a carefully thought out questionnaire
In The Seven Pillars of Statistical Wisdom book by Stephen M. Stigler:
William Stanley Jevons, writing in his Principles of ...
0
votes
3answers
54 views
What does level of significance mean in context of medical applications?
Suppose I have two drugs; A and B. B is a new drug that has to be introduced in the market.
$H_o\text{: Drug A and B are both equal}$
$H_a\text{: Drug B is better than Drug A}$
If I set my level of ...
1
vote
0answers
19 views
How to analyze repeated measures when condition changes at each time point?
I have a dataset from a repeated measures experiment that I am trying to analyze. The experiment had 4 possible conditions. Participants were measured on 1 condition and then again on a second ...
0
votes
0answers
29 views
When and when not to use an A/A test?
I'm curious about the circumstances under which an A/A test is a appropriate, vs. when it is not.
Here is my current understanding:
A/A testing is an empirical method, and the point is to ...
0
votes
0answers
5 views
Manipulating neuropsychology test items, controlling for changes?
If I take a subtest from an existing neuropsychological test, and take out 5 items to use independently as my methods of measurement for each level of my within-subjects factor, how can I ensure good ...
3
votes
1answer
70 views
Biased estimator obtained by optimal experiment design
I am using a model-based approach to infer the parameters of a given system. Namely, I represent my system by a model $\mathcal{M}$ with parameters $\theta$. To estimate the true value of $\theta$, I ...
1
vote
0answers
24 views
Best approach for maximising power?
This seems like a relatively very simple question compared to the apparent complexity on this forum.
I'm putting my PhD method together, within-subjects experimental design (counter-balanced related ...
