# Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

1,424 questions
Filter by
Sorted by
Tagged with
56 views

### D-Optimal Criteria Vs Differential Shannon Entropy

I want to understand how minimizing the determinant of the information matrix is equivalent to maximizing the differential Shannon entropy? A similar question was posted in Math SE but hasnt been ...
152 views

### Bayesian A/B test analysis for non-binary actions

When analyzing experiments with binary action (for example, will a user convert or not), then Beta-Binomial distribution is apt for that. For example, to model Click Through Rate (CTR): ...
38 views

### Do I have to randomize all drug trials?

This thought experiment I came up with has been bugging me. Let's say I want to test the effectiveness of Drug A vs Drug B on reading speed (no placebo necessary). I KNOW for a fact that the effects ...
27 views

### Conditional treatment effects for RCTs

I have randomized controlled trial data with one control and two treatments. Since the DV is continuous, I estimate the treatment effect using an lm model, while ...
22 views

### Why confounding factor has to correlated with independent variable?

One assumption of linear regression is the independent variables should be independent of each other. But in experimental design, when we build a linear regression to measure the causal impact of the ...
11k views

### What's the difference between a randomized block design and two factor design?

What's the difference between a randomized block design and a two factor design? They both use two-way ANOVA, and your blocks can be your factor.
138 views

### What is the meaning of “each treatment ” in BIBD?

(I am restating the question again in a more clear manner, as it was not clearly written last time) My doubt is regarding Balanced Incomplete Block Design (BIBD) Consider the following design with ...
40 views

### How is it decided to use Difference in Differences, Regression Discontinuity or both together (if possible) according to the policy change?

I am quite new at (quasi) natural experiment analysis, so please bear with me, and got some questions. In my research, I am planning to use one policy change which happened 2002 and treated firms are ...
132 views

### Partially crossed design and how to apply the mixed model?

I have a list of patient samples that were tested at 3 clinical sites; for site 1, reagent lots 1 and 2 were applied; for site 2, reagent lots 1 and 3 were applied; for site 3, reagent lots 2 and 3 ...
30 views

### Can a “crossed” experimental design compare a continuous and categorical variable?

Would it be incorrect to refer to an experimental design as "crossed" if one of the treatments is a continuous variable rather than a categorical variable? The continuous variable covers roughly the ...
60 views

### Optimal unbalanced design for A/B test

I want to find the right way to distribute variants (control/test) in A/B-test with the constraint of no more 10 percent of population in test group. The test is t-test for continuous metric. I can ...
27 views

### Apply normality assumption on limited sample size

I now have a measurement which only gives me: i) the range of values (continuous in nature), say, 110.0 to 160.0 without the access to the actual underlying data from which these statistics are ...
42 views

### What to do if the range of an input factor changes after running the experiment?

I am gaining my first experiences in Designing Experiments and here where I am stack: An experiment has been run with some continuous input factors: x1, x2, ..., x5. The goal of the experiment was to ...
17 views

### What kind of test for percentages of different groups?

I've set up a little experiment using 5 different treatments (including the control group) for control of crown rot in harvested bananas. For each treatment, I took around 18-20 banana clusters and ...
5k views

### Famous easy to understand examples of a confounding variable invalidating a study

Are there any well-known statistical studies that were originally published and thought to be valid, but later had to be thrown out due to a confounding variable that wasn't taken into account? I'm ...
26 views

### Sample Size Calculation considering Gradual Improvements to Treatment Group

I have an experiment with two equally sized groups. These groups contain machines and the metric being measured is the failure rate of the of the groups. In a typical experiment setting, the test ...
27 views

### Which additional experiments would best improve the model?

I have been given some data on a biotechnological process stemming from a multitude of different (kind of independent) experiments. The process has a lot of potentially relevant inputs (temperature, ...
26 views

### experimental design and linear regression

In an experimental design, randomly assigned participants into treatment group and control group and there is no statistical difference(using t-test) between two groups. The treatment group was ...
16 views

### Sample size calculation for AB testing in offline scenario

I have sales data for 500 stores and lets say I want to run an AB test on a sample of test and control stores with the objective of getting statistically significant increase in sales(lift). My ...
37 views

### List Experiment “ictreg.joint” error: “log-likelihood is not monotonically increasing”

I'm trying to use "ictreg.joint" function in "list" package in R by Imai and Blair to use the predicted responses from the list experiment as the explanatory variable to analyze an outcome variable. ...
154 views

### Using a model to select between a/b in a a/b test

I am wondering what the best way is to model an A/B test. Meaning, if I run an a/b test, I want to run a model to determine if A or B is better for each individual. A/B would be assigned randomly (50%...
21 views

### MDE minimal detectable effect

I m not able to figure out how to calculate MDE(minimal detectable effect) when designing an experiment. I see that there are calculators to do it for you like this one: https://bit.ly/2XgMjld For: ...
5 views

### Interaction Effects, Zero or non-Zero in 2 factor factorial design?

In a 2 factor factorial design it is common to use the following linear model and the associated notation: $y_{ijk}=\mu+\alpha_{i}+\beta_{j}+(\alpha\beta)_{ij}+\epsilon_{ijk}$ $(\alpha\beta)_{ij}$ ...
22 views

### detecting correlation between and significance of non-linear model parameters

I have a set of observed data y,obs that I am modelling using a non-linear model (currently in EXCEL though I also have a Python model). There are 4 input ...
67 views

### How to include an interaction with sex

I am performing linear regression on a dataset and want to include an interaction between sex and another covariate. However, in my model sex is coded as $0$ for female and $1$ for male. If I have ...
39 views

### How should I analyse the results of my experiments with respect to their hyper-parameters?

I trained (and tested) a CNN with the same architecture on the same task (MNIST). However, I trained it several times: one for each combination of hyper-parameters. Each model is trained and tested ...
181 views

### How to choose/select an experimental design

Is there a rule for selecting which experimental design one should carry out? I am new to statistics/ED and sometimes I find myself asking which design is best to use. My general guidelines have been ...
145 views

### Different groups, same intervention. What design is this?

I'm trying to figure out the best way to analyse the following clinical trial: Group1: Healthy individuals Group2: Diseased individuals Both groups follow the same intervention and are tested pre- ...
10 views

### How can I apply a design effect of 2 on my fisher's test and/or regression?

I'm trying to use R or statistical analysis on a multicluster sampling and specifically to inform it that I have a design effect of 2. I've familiarised myself with the survey package and tried to ...
19 views

### How to deciding on the number of experimental runs in R for orthogonal fractional factorial design?

Presently I am working on designing a questionnaire for my discrete choice experiment. I want to generate an orthogonal fractional factorial design for the following problem- The respondent has to ...
35 views

### Experiment design advice

Iād like to understand a good experiment design for the following example: Say sales people are trying to find the ideal time (in days) to send a follow up phone call to increase conversion (a sale). ...
106 views

### How to analyze factors that have missing by design values for certain observations?

I am using an analysis for an experiment that I consider suboptimal. I am asking for any suggestions to improve this. I will briefly outline the experiment and what Iāve done, then ask your advice. ...
40 views

### Generating a full rank model matrix in R

For a particular tool I need a model matrix which allows me to build desired contrasts and which is a full rank, i.e. no columns are linear combinations of other columns. The experimental design ...
10 views

### Factor analysis with missing data

I am looking for methods that allow me to do an exploratory factor analysis with missing data. The reason I need to account for missing data is that we will try to collect a large number of variables ...
25 views

### How to interpret coefficients of a cross-lagged panel analysis

I'm trying to understand the coefficients of this cross-lagged panel analysis. Are they correlation coefficients? are they regression coefficients? I'm not sure how can I interpret them if I have no ...
22 views

### Compare performance of libraries - test selection

I am trying to perform a comparison between three implementations of the same algorithm but using different libraries. However, I am not sure about my methodology. I am measuring elapsed time as a one ...
17 views

### $\mathbb{G}=(1,B,T,B \times T)$ is an orthogonal design

I have an orthogonal system with 2 factors, B and T. I know that B and T are orthogonal geometric. But I also need to show that that the system $\mathbb{G}=(1,B,T,B \times T)$ is an orthogonal ...
8 views

28 views

### Matched Pairs design definition

A horticulturist is planning a greenhouse experiment with 20 individually potted tomato plants to compare two fertilizer treatments. Ten plants are randomly assigned to fertilizer A, and the remaining ...