Questions tagged [experiment-design]

The study of how to structure an information-gathering exercise where variation is present.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Exogenous but non-random event for DiD

I have a quick question regarding whether the following event might pose any issues for my study. I am interested in exploring perceptions of climate change. In the early 1970s, former dictator Park ...
chunguc1004's user avatar
3 votes
0 answers
22 views

Regression models as goal metrics for experiments

What are the known issues or pitfalls associated with scoring regression/ML models on observations involved in a randomized experiment (a/b test, etc.)? In a randomized experiment, for most metrics (...
user1993951's user avatar
0 votes
0 answers
6 views

expected number of events under H0 and under alternative [closed]

Imagine that I have a trial design with interim analysis. What is the expected number of events under the null hypothesis? Under the alternative? Could you please help me to understand what it is ...
Kate's user avatar
  • 93
0 votes
0 answers
14 views

Machine Learning Experiment: should training parameters be fixed for a valid comparisons across model?

I am training an autoencoder three times, each time on a different dataset. The three datasets all have the same number of features, but have vastly different sizes. Assuming one of the datasets is A ...
BovineScatologist's user avatar
0 votes
1 answer
44 views

Usefulness of Latin Square Design

I want to understand the usefulness of the Latin Square Design. Suppose there are two block factors, each with three blocks. There are three experimental groups: A, B, C. One realization of the Latin ...
quibble's user avatar
  • 1,436
0 votes
0 answers
10 views

Is there a best approach for statistical power for field experiment?

I'm setting up a field experiment and want to make a solid plan for the analysis, since that's whereI get stressed in the process. I will deploy 26 pairs of traps. 13 pairs will be in urban forest, ...
E10's user avatar
  • 81
1 vote
0 answers
14 views

Is a permutation test valid in comparing a fraud block policy false positive rate across two different months?

I'm looking to conduct a counterfactual experiment in an e-commerce fraud context. When a fraud policy blocks a transaction you necessarily will never know if it was truly fraudulent. I'm trying to ...
phx's user avatar
  • 11
0 votes
0 answers
17 views

How can I implement a linear mixed model on data where the reference doesn't have repeated measures?

I want to analyze in paired data that has many dependent variables (continuous) and several categorical predictors. The predictors are: Group (patient/control), Treatment (none, t1, t2, t3), and Gene (...
maglorismyspiritanimal's user avatar
3 votes
1 answer
186 views

Reduce the model sequentially

I was given an ANOVA table and asked to reduce the model sequentially. I searched the online resources say: When reducing the model sequentially, you typically start by assessing the significance of ...
Matata's user avatar
  • 671
2 votes
1 answer
44 views

Statistical Test or Regression for Intervention Effectiveness?

I'm looking to model the effectiveness of an intervention on human behavior. I have control groups and treatments groups for the outcome of interest and pre and post measurements (just 1 for each) for ...
Statsnoob's user avatar
1 vote
0 answers
18 views

Why can't interaction be measured with a complete block design?

I am struggling to understand why interaction cannot be measured in a CBD. While many sources say it is because there is only one observation per block/treatment combination, could you not compare the ...
Laura Edwards's user avatar
0 votes
0 answers
32 views

Looking for an intuitive explanation of D-Criterion for Optimal Design Problem

I know only a little about Fisher information and optimal experimental design, but I'm trying to better understand the subject. If I have an experiment composed of a single detector and my detector ...
David G.'s user avatar
  • 103
0 votes
1 answer
22 views

Show conditional independence in randomized experiment

I'm new to causal inference, and I have a question regarding a potential causal relationship. I'm wondering if the following assumption holds: $(Y_i(1), Y_i(0), X_i) ⊥ Z_i -> (Y_i(1), Y_i(0)) ⊥ Z_i ...
user408381's user avatar
2 votes
1 answer
37 views

Regression using unordered combinations, sign of predictor depends on order

First time poster here. Is there anything wrong with the following regression approach? Background: I am dealing with fuel consumption $F$ of a vehicle under a variety of conditions. I have developed ...
Evan Stegner's user avatar
0 votes
0 answers
25 views

What is the correct way to analyze this data in a mixed model approach?

I have an experiment with rates of an waste in complete randomized blocks designs, with evaluation of soil fertility and tree heights/diameters over the years. Which is the best model to fit this data?...
ntrentin's user avatar
7 votes
2 answers
582 views

average revenue per user increases each country but decreases when combined

I'm trying to figure out what could be some reasons when we do the AB testing we're seeing average revenue per user increases for each country but when we calculate the overall average revenue per ...
ggggogloria's user avatar
3 votes
1 answer
81 views

How is it reasonable that randomised controlled trials can be used to perform causal inference?

I understand that randomised controlled trials (RCTs) are used to perform causal inference, but I'm a confused about how this is reasonable. Let's say that we have a treatment, and we want to find out ...
The Pointer's user avatar
  • 1,812
0 votes
0 answers
13 views

Factors in a multi-factor design

I have trouble distinguishing between a factor and a replicate. For instance, in my class notes, there is a question: To study the variability in CO2 emission rate by global regions 4 countries: US, ...
Matata's user avatar
  • 671
0 votes
0 answers
22 views

How to handle multicollinearity with varying length and type of conjoint treatments?

We ran a complicated experiment and are struggling to build a linear model that estimates everything we're interested in. We showed each person a description of a product (for illustration, let's say ...
Tia's user avatar
  • 51
2 votes
1 answer
90 views

Experiments where the treatment is such that the act of applying the treatment to some people within the group and not others undermines the treatment

Let’s say we have an experiment where we want to apply some treatment to some group to test whether the treatment “works”. But the treatment is such that the act of applying the treatment to some ...
The Pointer's user avatar
  • 1,812
1 vote
0 answers
21 views

How do I calculate effect size in a pretest-posttest three group randomized study for two planned contrasts (ANCOVA)?

I am having a hard time calculating the individual effect sizes for ANCOVA simple contrasts ($1$ vs. $2$ and $1$ vs. $3$) in a randomized pretest-posttest design study with three groups. In the ANCOVA,...
Eniko's user avatar
  • 11
2 votes
1 answer
55 views

Can regression to the mean also impact heterogeneity?

I understand Regression to the Mean (RM) as a naturally (and likely randomly) occurring shift in the scores of a sample of extreme subjects (super high or low scorers) from each measurement time to ...
Simon Harmel's user avatar
4 votes
1 answer
127 views

how to deal with treatment dropouts in experimental designs

I conducted a between-subjects experiment with one 3-level factor (high group vs low group vs control group). Because of dropouts from the treatments, the final distribution is control group: 85 ...
dondu's user avatar
  • 41
5 votes
1 answer
79 views

Performing a Difference-in-Difference Analysis where the Control Group is Already Treated

I'm involved in a project where the outcome is the proportion of cancer patients who have received surgery. The treatment event is a state-level policy change that mandated moving all these cancer ...
user8408275's user avatar
0 votes
1 answer
60 views

How to Include a 'No Shift' Scenario in a Temperature Shift DOE

I'm currently working on a Design of Experiments (DOE) for a process that involves temperature shifts at three different times. My challenge is incorporating a 'No Shift' scenario as one of the ...
Megazord's user avatar
2 votes
1 answer
67 views

How do data/observations themselves contribute to our ability to make determinations of whether a hypothesis is true or false?

Let's say that I have a hypothesis, and I'm trying to determine whether the hypothesis is true or false. If I remember correctly, in statistics, the standard philosophical view is that, technically, ...
The Pointer's user avatar
  • 1,812
2 votes
1 answer
80 views

Is it possible to use simulated (bootstrap) A/A tests of historic data to estimate the impact of confounding factors on the treatment effect?

I recently heard a proposal for a method to measure the degree to which confounding variables impact historic results of A/B tests. In order to ascertain the degree to which confounding has impacted ...
McGez's user avatar
  • 91
1 vote
1 answer
38 views

How to code a lme, lmer, or glmer model for a split-plot design (ANOVA)?

I have a study where the initial height of seedlings were measured at year 0 and again 11 years after treatments were applied and a net change in height was recorded. I have a split-plot design as ...
kryogenic7's user avatar
2 votes
1 answer
78 views

Power analysis for a quasi-experimental factorial design 3(manipulated) x 1 (measured) ANCOVA

I'm experiencing an hard time looking for the proper analysis to use to compute the power for an experiment i'd like to implement, so here is my question for you. I'd like to run a quasi-experimental ...
GT87's user avatar
  • 143
0 votes
0 answers
23 views

What is the appropriate design for a discrete choice experiment?

I need to develop a discrete choice experiment in R, which is an unlabeled experiment. However, after delving into the theory behind discrete choice experiments, it remains unclear whether there is a ...
Amy95's user avatar
  • 1
1 vote
0 answers
32 views

Discretization in regression, experimentation, and causal inference as deafult [duplicate]

It crossed my mind that when designing an experiment and you're not interested in NHST but full regression model where coefficients for treatment exposure and relevant covariates are desired, perhaps ...
jbuddy_13's user avatar
  • 2,910
17 votes
3 answers
760 views

Causal diagrams necessary in randomized controlled trials?

I understand how a properly administered RCT rules out confounders because there are no variables influencing the treatment/control group assignment except randomization (meaning no backdoor paths ...
RobertF's user avatar
  • 6,024
1 vote
0 answers
59 views

How to mathematically prove the "transitive property of nested predictors"?

QUESTION I am studying the structure of experiment data sets, and I want to propose a rule that I call the "transitive property of nested predictors". The general idea is that… if there are ...
Chris Science's user avatar
0 votes
0 answers
49 views

Validating my experiment design and Mann-Whitney U test usage

There are four different methods I use to generate solutions for different domains. To determine which solution is the most useful, I want to conduct an experiment with participants. As I have some ...
Max Bragilovski's user avatar
0 votes
0 answers
30 views

Caclulating the standard error of difference in Conditional Average Treatment Effects (CATEs)

Some perceive scientific information (for example, scientific evidence of climate change) as accurate, but others don't. I want to know under which condition this biased evaluation increases or ...
Jin's user avatar
  • 1
1 vote
0 answers
43 views

Which field of mathematics should be used to formalize the analysis of a dataset's "geometric" structure [closed]

I want to know if mathematical formalism can be applied to understand the "geometric" structure of an experiment's data set, and esp. to the relationships between its categorical predictors. ...
Chris Science's user avatar
2 votes
0 answers
29 views

Using Difference-in-Differences With No Natural Groupings

I'm running across a number of analyses at my organization where people are doing difference-in-differences on individual-level data with no natural groupings. For example, one looked at the effect ...
kypc's user avatar
  • 21
8 votes
1 answer
130 views

Is there a good reason for a lab to repeat experiments instead of conducting a single larger blocked experiment

It is not uncommon for biologists to repeat experiments to confirm initial results. Intuitively this makes some sense but to me seems inefficient and potentially problematic. (For example, one client ...
George Savva's user avatar
  • 2,259
0 votes
0 answers
35 views

Appropriate design for two experimental conditions and incomplete blocking

I'm planning to perform an experiment in which I will be assessing the role of two experimental variables (A: 6 levels, B: 4 levels) on a defined outcome which, however, might be affected by external ...
Mabri's user avatar
  • 21
0 votes
0 answers
8 views

Conceptual question on Recognition vs conformation

So, this is a conceptual question on the design of a model. I am getting started with ML and on a slight time crunch to get some results. My problem is not a classical recognition problem. It’s more ...
H Khalid's user avatar
5 votes
0 answers
22 views

Use of other ICU patients as control group

I am reviewing an article that is not yet published, so I cannot be too specific. Briefly, the authors did an unmatched case-control study. The cases were patients admitted to the ICU for a specific ...
Peter Flom's user avatar
  • 118k
0 votes
0 answers
25 views

Testing correlations between different varibles from two measurements

The field study investigates the relationship between root traits and plant performance traits in 12 genotypes, using a randomized complete block design with four replications per location. Root ...
Ales Kolma's user avatar
0 votes
0 answers
27 views

How to choose the sample size

How do I choose a sample size for a multiple factors irrigation experimental research? Here is what I mean, I want to compare the effect of precision irrigation and conventional irrigation on crop ...
Salami Chakour's user avatar
1 vote
0 answers
23 views

Mixed effects model in R with subsampling in replicates

I am working on a mixed effects model in r where I have 2 treatments, 5 replications and subsampling with each site. I dont have access to the raw subsampling data though, I just have the mean effect ...
Caleb Niemeyer's user avatar
0 votes
0 answers
12 views

How to adjust blocks in a 2x2 factorial experiment (Minitab)?

I'm currently in the planning phase of an experiment involving 2 factors, each with 2 levels, and 3 replications per combination, resulting in 12 experimental runs. In Minitab, the suggested blocks in ...
Sina Ardalan's user avatar
6 votes
2 answers
125 views

Designing an experiment to assess the form of a distance measure

BACKGROUND: I do research in budget-proposal aggregation: each person reports their ideal budget-allocation, and a central authority has to decide on an actual budget-allocation. An important ...
Erel Segal-Halevi's user avatar
4 votes
1 answer
26 views

AB Testing with and without counterfactuals

My boss wants us to run a simple experiment where we manipulate the search results on our website. He is insisting that we must have counterfactuals for the results to be valid. The way I see ...
Daniel's user avatar
  • 221
2 votes
0 answers
44 views

Validating power analysis

I'm currently running some experiments in web. The main idea is to validate the impact in some conversion metrics. The typical funnel we have is: Traffic -> Signup -> some Activation event. Our ...
marz's user avatar
  • 187
4 votes
1 answer
68 views

How does G*Power calculate sensitivity in z-test for two independent proportions

I am trying to recreate sensitivity calculation in Python by applying the same methodology as used in G*Power z-tests > Proportions: Difference Between Two Independent Proportions > Sensitivity: ...
Eugene Krall's user avatar
0 votes
0 answers
9 views

What is the relationship between maximum entropy sampling and bayesian d-optimal design?

In regards to optimal design, specifically sequential design or active learning for parameter inference (e.g. linear regression). As I understand it, Bayesian D-optimal design maximises the expected ...
Septimus Boshoff's user avatar

1
2 3 4 5
40