Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

Filter by
Sorted by
Tagged with
10
votes
2answers
9k views

AB test sample size calculation by hand

Evan Miller has created a well-known online AB test sample size calculator. For the sake of being able to program and modify this formula, I would like to know how to calculate sample size Evan Miller-...
4
votes
2answers
41 views

Power analysis for unknown experiment duration

How would I do a power analysis if I do not know how long we want to run the study? I want to use analysis to inform how long I want to run the experiment based on how long it will take to gather the ...
2
votes
1answer
31 views

Why do discretised predictors have lower statistical power than continuous predictors?

In designing an analysis, I'd like to decide between using discretised variables versus using the original, continuous variable (the reason being that in this particular case, collecting discretised ...
0
votes
0answers
14 views

How to choose the right analysis for a partially nested design?

I am not even sure if it is appropriate to call this scenario a partially nested design, but I settled for it for lack of a better term. My design is similar to the following in terms of its overall ...
1
vote
1answer
23 views

Experimental Design for Equivalence Testing

I'm working on setting up a study that will test how quickly users click on various icons appearing on a computer screen using three different prototype mouse types. Subjects will visit on three ...
1
vote
0answers
13 views

How could analyze this type of problem? [closed]

I had a problem the other day where we were asked to estimate the reduction to damage to the road and safety violations from increasing the number of inspectors? I could not think of any good ...
0
votes
0answers
35 views

How to decide the right sample size in A/B testing

I'm working on a price optimization problem where I want to change the price of a product and see if the revenue becomes higher. So, I have 40 different stores in my experiments and I split them into ...
2
votes
1answer
38 views

Deriving the expectation of SSE in randomized block design

I am trying to derive the expectation of the $\operatorname{SSE}$ in a randomized block design, $\mathbb{E}(\operatorname{SSE})$, with $$\operatorname{SSE} = \sum_{i=1}^b \sum_{j=1}^k(y_{ij} - \...
0
votes
0answers
11 views

What is the correct data structure for a Split-Plot design?

I was reading about split-plot designs and found some references that say the blocks (i.e. main plots) are 'nested' within the main plot treatment factors (left), but other references that say the ...
0
votes
1answer
20 views

Significance Testing between marketing campaign variants with only constant spend

I'm testing two marketing campaigns against each other: the assets are exactly the same, but the main difference is the optimization event that I'm feeding the vendor (e.g. Snapchat/Facebook/etc.) The ...
0
votes
0answers
4 views

Data setting in Intention to Treat estimates

Do we need to consider the baseline data while computing ITT estimates? Recently I came across the code for calculating ITT estimates of an RCT study. It ran something like this Reg y assign_treatment ...
0
votes
1answer
22 views

Difference-in-differences where treatment is applied multiple times

I am working on a project and I would like to try a difference-in-differences approach. I want to study how increases in network affect some outcome variable. Basically, I defined "treatment"...
3
votes
0answers
27 views

Advice for conducting the same experiment several times?

I'm very new to Bayesian statistics, but I've found myself in a situation where they might be able to help. I'll be running the same experiment repeatedly and analyzing the data every few months. The ...
1
vote
1answer
358 views

Help with modeling GRBDs experimental design analysis (Generalized randomized block design)

I am struggling with the formulation in lme of a generalized randomized block design (GRBD's) with subsampling. The experiment consists of 2 treatments: Genetic ...
3
votes
1answer
50 views

Designing an experiment with linear regression analysis: more categories or more observations per category?

During the design of my experiments I encountered the following question: When doing a linear fit (least squares), does it make more sense to have more categories or more observations per category. In ...
1
vote
1answer
24 views

Experimental Design: Choose Data Points to Minimize Quadratic Term Variance in Multiple Regression

$\newcommand{\eps}{\varepsilon}\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Suppose that you wish to fit a model $$Y=\beta_0+\beta_1x+\beta_2x^...
0
votes
0answers
7 views

Pseudo-replication in correlative studies

I am conducting a study. We are looking at soil properties across five different soil types. We have a map of polygons for the different soil types. We are not manipulating the soils, therefore, I ...
5
votes
1answer
2k views

T-test for regression coefficients obtained from Ridge, LASSO etc

In ordinary least squares, for example in an experimental design case, I obtain the regression coefficents by: $ \hat B = {({X^t}{X})}^{-1}X^ty$ Then, my null hypothesis for each coefficent is: $...
0
votes
0answers
12 views

How to understand the results of RDD(Discontinuous regression design)

what's the meaning of the 3 p-value?
2
votes
1answer
692 views

Difference between confounding and aliasing in 2^k factorial design

In statistics, particularly in experimental design, what is the difference between confounding and aliasing in 2^k factorial designs? Also how is a principal block related to the two concepts? I've ...
0
votes
0answers
19 views

Joint or separate regressions when groups have independent variables with distinct values

I have three independent variables x1, x2, x3 which are proportions summing up to 1, so I am only using x2 and x3 as the independent variables. The dependent variable is 3 level categorial variable. I ...
0
votes
0answers
18 views

Appropriate Way to Measure the Influence

I have two online marketing channels A & B and have their sales data for the past 24 months, as below. Total sales (conversions) are measured as A+B. B is the cheaper channel and hence my ...
31
votes
4answers
13k views

Independent variable = Random variable?

I'm slightly confused if an independent variable (also called predictor or feature) in a statistical model, for example the $X$ in linear regression $Y=\beta_0+\beta_1 X$, is a random variable ?
2
votes
1answer
43 views

How to tell if a mixed-model will be inherently unidentifiable

I am analysing data from an online survey measuring medical use of illicit and licit cannabis. There are three groups of users: those who use prescribed medical cannabis only, those who use both ...
0
votes
2answers
164 views

How to target data gathering to minimize loss-function?

I have a data-set, a model (single variable) and a loss function. I can collect more data but each data point requires significant analysis to obtain. Hence how can I target the data collection to ...
0
votes
1answer
23 views

ANOVA or T-test for multiple groups, each group with a set of observations?

Can someone please advise on which sort of experiment or test I should run for the following. I want to know which messages would potentially resonate more within a population. I have conducted ...
0
votes
1answer
81 views

Sample size and a study design: AI in shadow mode vs physicians

My research group has a group of patients where we are comparing the performance of a Random Forest classifier for acute respiratory tract infections. We are designing a study where we compare the ...
0
votes
1answer
33 views

Does a half-crossover design exist?

I've read about incomplete crossover designs, where subjects do not complete all treatments. However, I haven't seen anything about subjects completing all treatments, but not all treatment ...
0
votes
0answers
21 views

Can I regress overlapping independent variables?

give cash coupon give non-cash coupon Brand B Item A Item B Non-Brand B Item C Item D give cash coupon ratio= (item A Sales volume + item C Sales volume)/ total sales Brand B sales ratio = (item A ...
0
votes
0answers
11 views

RSM and steepest ascent for new $x'$ coordinates

In Myers, Montgomery, & Anderson-Cook, C. M. (2016)$^1$ the authors provide (p. 235) a general methodology to find new $x'$ coordinates in the path of steepest ascent based on constrained ...
1
vote
2answers
23 views

What methods exist to identify the optimal splits in some exogenous variable, such that a dependent variable is maximized?

I've come across an interesting problem recently, and I'm wondering if I'm missing some obvious approach here. The problem statement is thus: Imagine I run an online business, and I'm interested in ...
0
votes
0answers
16 views

How does two-step randomization affect the variance partitioning in an ANOVA?

I am trying to understand how multi-step randomization affects ANOVA results. For example, I can think of a pair of experiments, both with 2 crossed factors, FxG. In the first experiment, shown on ...
0
votes
0answers
24 views

Is there any formal statistical theory of social science research in which researchers often have to reinterpret the same data?

When the scientist can control variables and design experiments, standard statistics (hypothesis testing, etc) applies. If needed, the scientist can always redo an experiment to get more data. In much ...
0
votes
1answer
23 views

Matchit - Distance between 2 Propensity scores for findings specific pairs

helo I'm trying to find "spare" controls in case of drop-out in my primary control. I took my primary control sample (n=29), and ran Optimal match for find secondary match sample (N=350 ...
1
vote
1answer
29 views

Why is it acceptable to use the Sum of Squares of an Interaction as the Sum of Squares Error in a Randomized Complete Block Design?

Take for example the RCB design [Y = I + B + F + E], where... Y is the response I is the overall intercept B is a blocking factor with two levels F is a treatment factor with two levels E represents ...
0
votes
0answers
21 views

How to select control group if I have limited data points

We're launching a campaign and want to see the impact in consumer behavior. This is the experimental group. For choosing the control group I was thinking into checking for past data to see which is ...
1
vote
0answers
27 views

AB Experiment with Logistic Regression

I have built a model and need to test the effectiveness of this. I am planning to run an AB in the following manner Randomly assign population to Test Group (80%) and Control Group (20%) Within Test ...
0
votes
0answers
14 views

Determining the range of input feature values

I have an application (numerical simulation) in hand, for which I am generating my own data. Initially, I wanted to generate a small subset of data and perform initial ML model training. I was ...
0
votes
0answers
7 views

Which tests should i do with a control group and a experimental group with pretest-posttest design ( 2 levels)

I have an experimental design with a Control Group and a Experimental Group, i have one pretest and one posttest ( 2 levels i think i´ts called) for 5 different variables. Which tests should i run on ...
0
votes
0answers
33 views

Which analysis should I use on spss

There are 20 participants. Each participant is subjected to 4 conditions and each condition is repeated 9 times. Each condition yields 2 outcomes. I’m not 100% sure which analysis would be best used ...
0
votes
0answers
24 views

Randomized block design in R

I have a medical data from 5 different places. I was wondering how I could analysis the following question: Does using treatment increase mortality risk relative to using placebo based on conditions ...
0
votes
0answers
4 views

Terminology for Replications in Nested Models

My question is on terminology. Say you have an experiment in two locations, and say in each location you perform and RCBD with 5 Blocks in each location. Now for each location individually that means ...
7
votes
1answer
5k views

Case-control study and Logistic regression

Suppose we have case-control data, where cases have some disease ($Y$) and controls don't and we are interested in the association of some other variable(s) ($X$). I know that in this scenario we ...
2
votes
2answers
450 views

regarding generating synthetic data simulating the real data

We are trying to develop some predictive models. The current scenario is that we have to rely on synthetic data at first since the real data set will not be available quite soon. It is understandable ...
1
vote
1answer
205 views

Randomized Block Design Models

I have been asked to fit the following models in R: The models are based on this data: I know the mediums are the blocks and the units of the active ingredient are the treatments. I understand how ...
0
votes
1answer
7 views

Measuring how well our service detects malfunctioning parts with different percentages?

I am running an algorithm to detect malfunctioning parts of a system. We are simulating malfunctioning parts of the system with different percentages such as 10%, 25%,etc. What I want to show is how ...
0
votes
0answers
19 views

p-value for treatment/control ratio change in population

I have an experiment that has a slightly changing population as we drop individuals. Over the past 48 weeks, the treatment:control ratio has changed from 4:1 to 4.01:1. I want to assign a p-value to ...
1
vote
0answers
39 views

Sample Obtained is it Random Sample?

I feel embarrassed but I am confused with the below question. An interviewer plans to stand at an entrance to a popular shopping mall and conduct interviews. $$$$Is the below statement true? $$$$ ...
-1
votes
1answer
34 views

How do I derive the Sum of Squares sums for an arbitrary ANOVA?

I know it is possible to derive the Sums of Squares sums for a simple one-way CRD ANOVA. It is done by substituting the treatment factors into the total SS equation, then squaring, eliminating terms ...
0
votes
0answers
11 views

Individual Covariate in Fractional Factorial Design?

I want to make a marketing experiment (sending an e-mail) with fractional factorial design. I will send this e-mail to a mailing list consist of 660 members. I have 6 factors and use the 1/2 fraction....

1
2 3 4 5
34