Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Stat experiment question on when to sample control/treatment

I was asked this question in a recent interview but basically the interviewer asked about when to sample control/treatment for an experiment. My apologies in advance as my memory on the question may ...
0 votes
1 answer
22 views

Statistical test for correctness of expectation

First time posting here. Suppose I have a random variable $X$ and after various algebraic rewriting and calculus I have arrived at a closed form of $\phi = \mathbb{E}[X]$. However, I am not totally ...
  • 3
0 votes
0 answers
25 views

Help Needed for Outliers detection post paired T-test statistical test

I don't know if this is a standard way od doing things so open to any suggestions, basically I have done random sampling from my population to create 2 groups Treatment & Control. I also have few ...
0 votes
0 answers
14 views

building design matrix of random effects in a LMM

I am working in simulations of data following a LMM. The general formula of the model, as defined in most texts books is as follows: $$Y=X\beta+Zb+\epsilon$$ Where $b\sim N(0,D)$, $\epsilon \sim N(0,R)...
2 votes
1 answer
29 views

The correct treatment of censorship and observation periods in survival analysis using longitudinal observational data

I have longitudinal retrospective data and want to perform Cox proportional hazards regression. Still, I would be extremely grateful if the CrossValidated community could sanity-check my understanding ...
1 vote
2 answers
30 views

Identifying the best n points from a dataset

The problem I am trying solve involves identifying the best n data points from a dataset. For example if I have dataset with x1,x2,x3 independent variables and y1,y2 dependent variables I want to find ...
  • 121
0 votes
0 answers
31 views

Cluster standard errors on experiment group?

I am running a very simple experiment. There are two different experiment groups, participants are randomly assigned. They have to make a simple binary decision in the experiment. I have three control ...
0 votes
0 answers
5 views

Sample size for direct estimation of position bias in LTR

Imagine an AB test where control is my current recommender, which recommends 10 items. Treatment is a random shuffle of those 10 items. My goal is to estimate position bias for each position while ...
  • 83
0 votes
0 answers
21 views

Does Fisher exact test impose restriction on sampling design? [duplicate]

It is known that Fisher exact test assumes that all 4 margins of 2 by 2 table are fixed. $\chi^2$ test does not impose restriction on sampling design beforehand in general where I mean $\chi^2\sim\sum ...
  • 1,179
2 votes
1 answer
14 views

Difference of baseline data between groups NOT significant on pre-matching [duplicate]

In a retrospective observational study, I'd like to compare the efficacy of drug A vs drug B, and was considering propensity score matching on age, gender, history of diabetes mellitus, and history of ...
  • 23
0 votes
1 answer
24 views

A/B Testing Question - Power Analysis for Two Proportion Z Test

I'm having a hard time wrapping my head around experiment design and what appears to be a disconnect to experiment outcome analysis. If I am proposing a z test of two independent proportions (...
0 votes
0 answers
14 views

Labeled orthogonal design in R

I have a problem preparing labelled orthogonal designs in the R program. I have two labelled alternatives; Four attributes are common for both of them but levels are different One attribute is just ...
2 votes
3 answers
159 views

What does the phrase "adjusted by" mean?

"Matching of one group with specific exposures or outcome with a comparable group is frequently used in medical research to investigate the association between exposure and outcome." Ref ...
  • 1,759
3 votes
0 answers
140 views
+100

What kind of study design is this and what kind of statistical analysis should be suitable?

I plan to conduct a study in which respondents will be asked to solve total 6 puzzles together. Is there a specific study design (name) for this? The scenario would be: First, ...
1 vote
1 answer
26 views

Question on estimating (OLS) the ATE of RCT with multiple (2) treatments

Updated: I do not have enough points to comment so... Thank you Ben, you did interpret my question correctly. There are three treatment categories: control, A, and B. Thank you for clarifying that ...
  • 11
1 vote
1 answer
23 views

How to do pairwise comparisons for Kruskal Wallis test?

If the null hypothesis in the Kruskal Wallis test is rejected, How can we perform pairwise comparisons? Can we apply Dunnett's Test for that?
  • 11
1 vote
0 answers
25 views

Between Subjects Likert Analysis -- Treat as Ordinal or Continuous for Stats Test?

I have 4 groups in a between subjects design answering likert scale questions. I am wondering how to analyse the data. There are 20 questions for each user in each group of 30 (so 120 people all ...
0 votes
1 answer
20 views

Does this study design lend itself to a logistic regression?

I have a cohort comprising of individuals who participate in a weight-loss program and their weight measurements are collected longitudinally for 12 months (not necessarily at the same time points). ...
1 vote
1 answer
32 views

Regression Analysis of Time as Independent Variable in Experiment

we conducted an experiment on a sample of 30 persons. For every person, we observed the outcome variable every 15 minutes, so for each person 10 times. IV: Time (in Minutes) DV: Interval Scale [1;9] n ...
  • 13
2 votes
1 answer
63 views

Infer individual power contribution from team efforts

I like to infer the contribution of each rower in a crew boat from a number of races: 8 rowers are split repeatedly into two boats of 4 rowers each. The race over a distance leads to an estimate of ...
5 votes
2 answers
51 views

How to choose number of sites and number sub-sites within each site in planning stage of a study? (ecology)

I have a situation in which I am going to perform a study on live blue mussels. The research question is how the prevalence of mussels affects the prevalence of other animals living on the sea floor. ...
0 votes
0 answers
15 views

What should I report on a 3-way ANOVA with a significant 3-way interaction effect?

I'm using graphpad prism to analyze some research data. The study design has 3 factors (Sex x Cardiac Anomaly x Exercise workload) for a variety of parameters. I don't use 3way ANOVA very often (and ...
0 votes
0 answers
14 views

When to use standard deviation versus standard error in linear error propagation

I have a question about linear error propagation. Let's say that I want to use an equation to calculate n, where n = (PV)/(RT) (eq.1) I only take one measurement of P, and one measurement of T, but I ...
0 votes
0 answers
18 views

chi-squared for experimentation analysis - how to adjust for prior difference?

If using chi-squared test of independence to check if treatment group has affected the distribution of a binary outcome (e.g. click vs non-click), but there is also prior difference in click rate ...
  • 171
2 votes
0 answers
18 views

Flip test and control group in causal inference to get sufficient sample sizes for synthetic control

There was supposed to be a buy-one-get-one (BOGO) promotion for sneakers at 5,000 stores. Unfortunately, a glitch happened, and 100 of the stores did NOT get the memo about the BOGO promotion, so they ...
0 votes
0 answers
10 views

Design for retrospective standardized testing and employment outcome study

we (student group) have been given a project involving about 10 years of corporate employment data, including variables on events like hiring date, training outcomes, promotion, retirement, etc. Also ...
4 votes
1 answer
63 views

Does it ever make sense to use the sum of the pre and post measure means?

Say I quantitatively measure a specific trait of each of the students in a group so that each of the students receive a score. The average score of the group is $X_0$. After a treatment, I measure a ...
0 votes
0 answers
30 views

Converting uncertain labels to probabilities for training data

I am devising a data collection effort to measure audio perceptual discrimination, and then model it using machine learning. What I am trying to model is: "Can a typical human hear the difference ...
2 votes
2 answers
45 views

Book recommendations for Combinatorial Designs

I am well-acquainted to basic topics of experimental designs such as anova, ancova, crd, rbd, lsd, factorial design, robust design etc (as per the book of Dr. D. C. Montgomery). Now as per my recent ...
2 votes
1 answer
47 views

Analysis of a campaign holdout experiment

We have an experiment where we are splitting an audience into 4 equal groups of 2000 customers each. 3 will receive an email but 1 group won't received anything. The goal is to test if they will ...
  • 23
3 votes
2 answers
32 views

Determine if treatment effect is distributed equally across participants in a study or driven by a few participants

This is perhaps a malformed question, but I'll give it a shot. Let's say I have a pre and post measure of some outcome variable and a dummy variable treatment that ...
1 vote
2 answers
56 views

Correcting for pre-experiment bias in proportions test?

Say I have an obstacle course, which not everyone completes, though, globally, most do. I hypothesize that the treatment, drinking Gatorade, will cause an increase in the obstacle course completion ...
  • 1,728
0 votes
1 answer
21 views

Notations in the statistical analysis of experimental research

I am very new to experimental setup and have recently started to learn about designing experiments and analyzing the results. In the papers that I read there are notations that I am unfamiliar with. I ...
  • 221
2 votes
0 answers
22 views

How to study the impact of time in a mixed-factorial design?

I ran a study to compare 2 different computer interfaces (say A and B) meant to be used in an unfamiliar environment. I want to compare: How much effort participants took to complete a certain task ...
1 vote
0 answers
48 views

What layout is this?

A fleet of electric vehicles are to be investigated for the mileage performance per charge. Four vehicles have been chosen randomly from the fleet and four observations of miles were made for each ...
  • 11
0 votes
0 answers
32 views

Is "triple differences" and estimating "heterogeneous treatment effects" the same?

I am running a difference-in-differences (DiD) regression with a time and treatment interaction and I have a continuous outcome variable. However, I hypothesize that the DiD estimator can be moderated ...
1 vote
1 answer
54 views

Exercises for A/B testing/Experimental design

I'm applying for a data scientist job, which requires a lot of knowledge in A/B testing/Experimental design. But actually I'm not very familiar with it. For A/B testing I was recommended a great book ...
0 votes
0 answers
22 views

Two way Mixed anova or something else ? (small sample size)

I would like to ask some questions about appropriate statistical test for our study design. This study aims to evaluate the postural balance among parkinsonian patients. We have 3 groups, and in each ...
0 votes
0 answers
29 views

How do I use Lehr's 'rule of thumb' with a binomial distribution?

(This is a crosspost from data science stack exchange: https://datascience.stackexchange.com/q/113436/139117) Not a data scientist, but hopefully this is the right forum for this simple question: I'm ...
  • 1
0 votes
0 answers
29 views

Minimum sample size for AB testing on a proportion derived from an extremely rare event

I am running an online experiment on a proportion that is the rate of an extremely rare event. 95.5% of the observations have 0 for this event. The majority of the rest have 1 for that event. Hence ...
  • 76
0 votes
1 answer
45 views

Experiment design useful when objects change / will be replaced in each trial run?

I want to perform some studies on how material thickness, type of material, humidity and temperature influence tensile strength. While I was thinking about useful designs I was wondering whether a ...
  • 2,441
3 votes
1 answer
60 views

How to account learning effect in study which cannot be conducted simultaneously?

Suppose I want to conduct a trial with two treatment arms in each of which there are 50 patients. The patients in both arms are iid sample from sick population. I would like to compare treatment ...
  • 1,179
1 vote
1 answer
33 views

A/B experimental design with strata or blocks?

I have two approaches I wish to evaluate conversions (currently at ~2%). As in many situations, I cannot assume equal likelihood of conversions of instances which any number of individuals can be ...
  • 4,743
1 vote
1 answer
23 views

Adjust results for a systematic error

My model identifies objects with a specific quality in a large population. I tested it on a sample of 500 and found that it fails to identify about 7% of the objects (false negatives) due to some ...
  • 41
2 votes
1 answer
49 views

Multiplicity issues in clinical trials for multiple groups

Let's say we design a RCT for two different populations and want to test an intervention vs a control. It could be adults and children, male and female, etc. We have one primary endpoint to be tested ...
0 votes
0 answers
25 views

How should one re-formulate hypothesis after hypothesis testing or whether or not to discard data after significance testing or archive it?

@Nalzook summarized what he thinks Ronald Fisher did. Ask a question. Propose a null hypothesis based on the question. Do some experiments, and collect some data. Assuming the null is true, calculate ...
0 votes
0 answers
13 views

Name for expeimental design using randomization without replacement?

I have a between subjects design with 3 conditions A, B and C. My goal is to have 30 people in each condition. I have no other balance requirements other than the sample size being equal across ...
  • 415
0 votes
0 answers
11 views

How are the interactions between three-level factor and four-level factor calculated?

I would like to know how the interactions between three-level factor and four-level factor can be calculated. The textbook I have says how to calculate the following interactions. between three-level ...
  • 1
0 votes
0 answers
21 views

In linear regression, when sample size of one treatment group decreases, the power for testing the other treatment group vs common control also drop

Suppose we have design where we have three groups: two treatment groups and a common control. Suppose that when each of the three groups have sample size being 30, the power for testing whether ...
  • 983
1 vote
0 answers
33 views

Mixture desing experiment and regression

I have an experiment with four groups with multiple trials. In two groups individuals have the exact same trials during all the experiment, no variation In the stimuli were added. But in the other ...

1
2 3 4 5
37