Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

Filter by
Sorted by
Tagged with
3
votes
0answers
19 views

Extending external validation diagnostics to experiments with continuous treatment

Does the external validation diagnostic methods discussed in Stuart et al. (2011) (i.e., inverse propensity score weighted regressions) also apply to the experimental setting in which the treatment is ...
1
vote
1answer
20 views

Basic question about measurement errors in Design of Experiments

After reading and applying many examples of DoE I have a question that has been bugging me and haven't found an answer so far. Usually what we see in an example or publication is the design table with ...
0
votes
2answers
321 views

How to test for a synergistic effect of A & B with proportions of treatment effects, if I don't have the full 2x2 table?

I currently have a 2x2 table of proportions for treatment effects that looks like: \begin{array}{lll} & \text{Treatment } A & \text{Without Treatment } A \\ \text{...
0
votes
0answers
14 views

How can I write the response surface methodology (RSM) equation from coefficients including block effect?

I fitted a model by rsm function in R, and here is the code: rsm1 = rsm(x ~Block+FO(x1,x2,x3)+PQ(x1,x2,x3), data=t1.df) The coefficients, obtained from the rsm1$...
1
vote
1answer
45 views

How to Deploy Contextual Bandits in Online Experimentation Platform?

This question is about how to deploy contextual bandits(CMAB) in the context of web site optimization and online experimentation. I implemented contextual free MAB(MAB). When I run a MAB experiment, I ...
0
votes
1answer
23 views

My Design defines the Hypothesis test?

a conceptual question appear in my mind and i need help, in pratical therms, whats defines the statistical test its my hypothesis and my experimental design ? Using this example, if i am interrested ...
1
vote
0answers
21 views

Using Latin Hypercube Sampling with a condition that the sum of two variables should be less than one [closed]

I am running a model with 4 variables defined from 0 to 1 as inputs (say $x_i \in [0,1]$ with $ i=1,..., 4$), but 2 of these variables must satisfy the condition that $x_1 + x_2 \leq 1$. How can I ...
0
votes
0answers
19 views

What is the best statistical analysis to use for the below example (psychology research)

I am currently choosing the best statistical design for my Psychology MSc research. I am looking into the effects of BMI (underweight, normal, overweight), SES (Low, high) and Ethnicity (Categorical) ...
1
vote
0answers
16 views

Repeated measures design: Must each participant receive ALL 4 versions of a given stimulus?

I am creating a 2x2 repeated measures design where a given stimulus object has four possible versions (2 factors with 2 levels each create 4 unique object versions). In the ideal repeated measures ...
1
vote
0answers
21 views

Codebook/survey design and modeling: How to deal with a large share of intentionally missing values? [closed]

In my final data set I have a high proportion of missing data (1/4 are NAs, 35/52 variables have NAs), because of my codebook design: I only coded features (how things were discussed) that occurred in ...
2
votes
0answers
20 views

Pre-posttest x Fixed Effects Design

I am planning a 2x2x2 design which includes 1 within factor and 2 between factors. However, the within factor is a pre-test post-test design, meaning that the order of the two treatments of the within ...
0
votes
0answers
10 views

What to do if some conditions of DoE are hard or impossible to run?

Assume we have 3 factors with 2 levels each. If I understood it correctly, for small number of factors we can consider a full factorial design, in this case would be 2^3 full factorial design. What ...
4
votes
0answers
52 views

D-Optimal Criteria Vs Differential Shannon Entropy

I want to understand how minimizing the determinant of the information matrix is equivalent to maximizing the differential Shannon entropy? A similar question was posted in Math SE but hasnt been ...
2
votes
1answer
142 views

Bayesian A/B test analysis for non-binary actions

When analyzing experiments with binary action (for example, will a user convert or not), then Beta-Binomial distribution is apt for that. For example, to model Click Through Rate (CTR): ...
2
votes
1answer
38 views

Do I have to randomize all drug trials?

This thought experiment I came up with has been bugging me. Let's say I want to test the effectiveness of Drug A vs Drug B on reading speed (no placebo necessary). I KNOW for a fact that the effects ...
0
votes
1answer
26 views

Conditional treatment effects for RCTs

I have randomized controlled trial data with one control and two treatments. Since the DV is continuous, I estimate the treatment effect using an lm model, while ...
0
votes
0answers
22 views

Why confounding factor has to correlated with independent variable?

One assumption of linear regression is the independent variables should be independent of each other. But in experimental design, when we build a linear regression to measure the causal impact of the ...
5
votes
2answers
11k views

What's the difference between a randomized block design and two factor design?

What's the difference between a randomized block design and a two factor design? They both use two-way ANOVA, and your blocks can be your factor.
0
votes
0answers
15 views

What statistical tests should be performed to test effect of trials on response times where each subject has different number of data-points?

I am running a behavioral experiment where a number of subjects (=20 in my case) perform a simple cognitive task. The experiment consists of a fixed number of trials (say, 40 in my case). During each ...
1
vote
1answer
138 views

What is the meaning of “each treatment ” in BIBD?

(I am restating the question again in a more clear manner, as it was not clearly written last time) My doubt is regarding Balanced Incomplete Block Design (BIBD) Consider the following design with ...
1
vote
1answer
34 views

How is it decided to use Difference in Differences, Regression Discontinuity or both together (if possible) according to the policy change?

I am quite new at (quasi) natural experiment analysis, so please bear with me, and got some questions. In my research, I am planning to use one policy change which happened 2002 and treated firms are ...
0
votes
1answer
130 views

Partially crossed design and how to apply the mixed model?

I have a list of patient samples that were tested at 3 clinical sites; for site 1, reagent lots 1 and 2 were applied; for site 2, reagent lots 1 and 3 were applied; for site 3, reagent lots 2 and 3 ...
0
votes
2answers
26 views

Can a “crossed” experimental design compare a continuous and categorical variable?

Would it be incorrect to refer to an experimental design as "crossed" if one of the treatments is a continuous variable rather than a categorical variable? The continuous variable covers roughly the ...
3
votes
2answers
57 views

Optimal unbalanced design for A/B test

I want to find the right way to distribute variants (control/test) in A/B-test with the constraint of no more 10 percent of population in test group. The test is t-test for continuous metric. I can ...
0
votes
2answers
26 views

Apply normality assumption on limited sample size

I now have a measurement which only gives me: i) the range of values (continuous in nature), say, 110.0 to 160.0 without the access to the actual underlying data from which these statistics are ...
2
votes
1answer
41 views

What to do if the range of an input factor changes after running the experiment?

I am gaining my first experiences in Designing Experiments and here where I am stack: An experiment has been run with some continuous input factors: x1, x2, ..., x5. The goal of the experiment was to ...
0
votes
0answers
17 views

What kind of test for percentages of different groups?

I've set up a little experiment using 5 different treatments (including the control group) for control of crown rot in harvested bananas. For each treatment, I took around 18-20 banana clusters and ...
51
votes
11answers
5k views

Famous easy to understand examples of a confounding variable invalidating a study

Are there any well-known statistical studies that were originally published and thought to be valid, but later had to be thrown out due to a confounding variable that wasn't taken into account? I'm ...
1
vote
1answer
24 views

Sample Size Calculation considering Gradual Improvements to Treatment Group

I have an experiment with two equally sized groups. These groups contain machines and the metric being measured is the failure rate of the of the groups. In a typical experiment setting, the test ...
1
vote
1answer
27 views

Which additional experiments would best improve the model?

I have been given some data on a biotechnological process stemming from a multitude of different (kind of independent) experiments. The process has a lot of potentially relevant inputs (temperature, ...
0
votes
0answers
26 views

experimental design and linear regression

In an experimental design, randomly assigned participants into treatment group and control group and there is no statistical difference(using t-test) between two groups. The treatment group was ...
0
votes
0answers
16 views

Sample size calculation for AB testing in offline scenario

I have sales data for 500 stores and lets say I want to run an AB test on a sample of test and control stores with the objective of getting statistically significant increase in sales(lift). My ...
0
votes
0answers
5 views

How to proceed when the data created from using a fractional factional design gives unsatisfying models?

We've got 10 factors to investigate. Some are binary, some are continuous. Initially we created a 2-level Plackett-Burman design (using min and max for the continuous factors) with 16 experiments (...
0
votes
0answers
30 views

List Experiment “ictreg.joint” error: “log-likelihood is not monotonically increasing”

I'm trying to use "ictreg.joint" function in "list" package in R by Imai and Blair to use the predicted responses from the list experiment as the explanatory variable to analyze an outcome variable. ...
0
votes
1answer
144 views

Using a model to select between a/b in a a/b test

I am wondering what the best way is to model an A/B test. Meaning, if I run an a/b test, I want to run a model to determine if A or B is better for each individual. A/B would be assigned randomly (50%...
0
votes
1answer
18 views

MDE minimal detectable effect

I m not able to figure out how to calculate MDE(minimal detectable effect) when designing an experiment. I see that there are calculators to do it for you like this one: https://bit.ly/2XgMjld For: ...
0
votes
0answers
5 views

Interaction Effects, Zero or non-Zero in 2 factor factorial design?

In a 2 factor factorial design it is common to use the following linear model and the associated notation: $y_{ijk}=\mu+\alpha_{i}+\beta_{j}+(\alpha\beta)_{ij}+\epsilon_{ijk}$ $(\alpha\beta)_{ij}$ ...
0
votes
0answers
22 views

detecting correlation between and significance of non-linear model parameters

I have a set of observed data y,obs that I am modelling using a non-linear model (currently in EXCEL though I also have a Python model). There are 4 input ...
4
votes
2answers
66 views

How to include an interaction with sex

I am performing linear regression on a dataset and want to include an interaction between sex and another covariate. However, in my model sex is coded as $0$ for female and $1$ for male. If I have ...
2
votes
0answers
39 views

How should I analyse the results of my experiments with respect to their hyper-parameters?

I trained (and tested) a CNN with the same architecture on the same task (MNIST). However, I trained it several times: one for each combination of hyper-parameters. Each model is trained and tested ...
2
votes
1answer
170 views

How to choose/select an experimental design

Is there a rule for selecting which experimental design one should carry out? I am new to statistics/ED and sometimes I find myself asking which design is best to use. My general guidelines have been ...
0
votes
2answers
141 views

Different groups, same intervention. What design is this?

I'm trying to figure out the best way to analyse the following clinical trial: Group1: Healthy individuals Group2: Diseased individuals Both groups follow the same intervention and are tested pre- ...
0
votes
0answers
10 views

How can I apply a design effect of 2 on my fisher's test and/or regression?

I'm trying to use R or statistical analysis on a multicluster sampling and specifically to inform it that I have a design effect of 2. I've familiarised myself with the survey package and tried to ...
0
votes
0answers
19 views

How to deciding on the number of experimental runs in R for orthogonal fractional factorial design?

Presently I am working on designing a questionnaire for my discrete choice experiment. I want to generate an orthogonal fractional factorial design for the following problem- The respondent has to ...
1
vote
0answers
35 views

Experiment design advice

I’d like to understand a good experiment design for the following example: Say sales people are trying to find the ideal time (in days) to send a follow up phone call to increase conversion (a sale). ...
4
votes
1answer
106 views

How to analyze factors that have missing by design values for certain observations?

I am using an analysis for an experiment that I consider suboptimal. I am asking for any suggestions to improve this. I will briefly outline the experiment and what I’ve done, then ask your advice. ...
2
votes
1answer
32 views

Generating a full rank model matrix in R

For a particular tool I need a model matrix which allows me to build desired contrasts and which is a full rank, i.e. no columns are linear combinations of other columns. The experimental design ...
1
vote
0answers
10 views

Factor analysis with missing data

I am looking for methods that allow me to do an exploratory factor analysis with missing data. The reason I need to account for missing data is that we will try to collect a large number of variables ...
1
vote
1answer
23 views

How to interpret coefficients of a cross-lagged panel analysis

I'm trying to understand the coefficients of this cross-lagged panel analysis. Are they correlation coefficients? are they regression coefficients? I'm not sure how can I interpret them if I have no ...
1
vote
0answers
22 views

Compare performance of libraries - test selection

I am trying to perform a comparison between three implementations of the same algorithm but using different libraries. However, I am not sure about my methodology. I am measuring elapsed time as a one ...

1
2 3 4 5
28