Questions tagged [experiment-design]
The study of how to structure an information-gathering exercise where variation is present.
1,635
questions
0
votes
0answers
20 views
Is there any formal statistical theory of social science research in which researchers often have to reinterpret the same data?
When the scientist can control variables and design experiments, standard statistics (hypothesis testing, etc) applies. If needed, the scientist can always redo an experiment to get more data.
In much ...
0
votes
0answers
12 views
How does two-step randomization affect the variance partitioning in an ANOVA?
I am trying to understand how multi-step randomization affects ANOVA results.
For example, I can think of a pair of experiments, both with 2 crossed factors, FxG. In the first experiment, shown on ...
0
votes
0answers
12 views
Is it possible to measure design effect without drawing actual samples? [closed]
Is it possible to measure DEFF (design effect) without drawing actual samples?
Based on e.g. just the sample sizes?
https://en.wikipedia.org/wiki/Design_effect
0
votes
1answer
20 views
Matchit - Distance between 2 Propensity scores for findings specific pairs
helo
I'm trying to find "spare" controls in case of drop-out in my primary control.
I took my primary control sample (n=29), and ran Optimal match for find secondary match sample (N=350 ...
0
votes
0answers
19 views
How to select control group if I have limited data points
We're launching a campaign and want to see the impact in consumer behavior. This is the experimental group. For choosing the control group I was thinking into checking for past data to see which is ...
1
vote
0answers
23 views
AB Experiment with Logistic Regression
I have built a model and need to test the effectiveness of this. I am planning to run an AB in the following manner
Randomly assign population to Test Group (80%) and Control Group (20%)
Within Test ...
0
votes
0answers
12 views
Determining the range of input feature values
I have an application (numerical simulation) in hand, for which I am generating my own data.
Initially, I wanted to generate a small subset of data and perform initial ML model training. I was ...
0
votes
0answers
6 views
Which tests should i do with a control group and a experimental group with pretest-posttest design ( 2 levels)
I have an experimental design with a Control Group and a Experimental Group, i have one pretest and one posttest ( 2 levels i think i´ts called) for 5 different variables.
Which tests should i run on ...
0
votes
0answers
30 views
Which analysis should I use on spss
There are 20 participants. Each participant is subjected to 4 conditions and each condition is repeated 9 times. Each condition yields 2 outcomes.
I’m not 100% sure which analysis would be best used ...
0
votes
0answers
23 views
Randomized block design in R
I have a medical data from 5 different places. I was wondering how I could analysis the following question:
Does using treatment increase mortality risk relative to using placebo based on conditions ...
0
votes
0answers
4 views
Terminology for Replications in Nested Models
My question is on terminology. Say you have an experiment in two locations, and say in each location you perform and RCBD with 5 Blocks in each location. Now for each location individually that means ...
0
votes
0answers
18 views
p-value for treatment/control ratio change in population
I have an experiment that has a slightly changing population as we drop individuals. Over the past 48 weeks, the treatment:control ratio has changed from 4:1 to 4.01:1. I want to assign a p-value to ...
1
vote
0answers
27 views
Sample Obtained is it Random Sample?
I feel embarrassed but I am confused with the below question.
An interviewer plans to stand at an entrance to a popular shopping mall and conduct interviews.
$$$$Is the below statement true? $$$$
...
-1
votes
1answer
29 views
How do I derive the Sum of Squares sums for an arbitrary ANOVA?
I know it is possible to derive the Sums of Squares sums for a simple one-way CRD ANOVA. It is done by substituting the treatment factors into the total SS equation, then squaring, eliminating terms ...
0
votes
0answers
11 views
Individual Covariate in Fractional Factorial Design?
I want to make a marketing experiment (sending an e-mail) with fractional factorial design. I will send this e-mail to a mailing list consist of 660 members. I have 6 factors and use the 1/2 fraction....
0
votes
1answer
7 views
Measuring how well our service detects malfunctioning parts with different percentages?
I am running an algorithm to detect malfunctioning parts of a system. We are simulating malfunctioning parts of the system with different percentages such as 10%, 25%,etc. What I want to show is how ...
1
vote
1answer
23 views
How to measure how well our service detects malfunctioning parts with different percentages?
I am running an algorithm to detect malfunctioning parts of a system. We are simulating malfunctioning parts of the system with different percentages such as $10\%$, $25\%$ ,etc. What I want to show ...
1
vote
1answer
25 views
Why is it acceptable to use the Sum of Squares of an Interaction as the Sum of Squares Error in a Randomized Complete Block Design?
Take for example the RCB design [Y = I + B + F + E], where...
Y is the response
I is the overall intercept
B is a blocking factor with two levels
F is a treatment factor with two levels
E represents ...
0
votes
1answer
34 views
Calculating Sample Size During A/B
Have spent a good bit of time googling this and could use some thoughts.
If we started an experiment and did not properly set stopping conditions and are now seeing significant p-values that have held-...
2
votes
3answers
40 views
In a RCT, what are the risk of selection bias if no allocation bias is present?
In the Wikipedia page about Randomized Controlled Trials (RCT), the procedures section states that RCT minimize selection bias and allocation bias.
My understanding is that selection bias "occur ...
2
votes
1answer
17 views
What is a "split-body" RCT?
I am reading the RCT Wikipedia page, in particular the classification by study design.
There are described four types of RCT: (i) parallel groups (ii) crossover (iii) cluster (iv) factorial.
Then ...
1
vote
0answers
15 views
2 way mixed within group and between groups ANOVA on unbalanced plan vs paired T-test
I have a data set of race results and I would like to compare speed difference between male and female.
$H_0$: The speed difference between male and female decrease as the race distance increases.
The ...
0
votes
0answers
27 views
Choosing correct Sample Size for pretest-posttest-control group design
I want to calculate the effect size for a pretest-posttest-control group design and the variance of the effect size.
Here are the equations for $d_{ppc2}$ and the variance of $d_{ppc2}$ from Morris (...
0
votes
0answers
65 views
Use of null hypothesis significance testing in observational data
My question is about the use and interpretation of null hypothesis significance tests in cases where we do not have random samples or designed experiments, but rather have observational data from the ...
0
votes
0answers
13 views
Analysis sense check: preregister planned contrasts which we wish to conduct even if associated ANOVA is nonsignificant?
I could use a sense check on my analysis plan for an experimental study which has not yet been run; I will preregister it once I am confident that it is reasonable. There are seven between-...
1
vote
0answers
36 views
Generating realistic mock data for a real-world scenario
Im a final year math student doing a project for a company. They want to do a cup-and-grid water drop test with a firefighting helicopter. The basic idea is: a grid of 1000cups is setup on an airfield,...
0
votes
0answers
18 views
How many types of randomization are there? And how they each dealt with in the experiment's design or statistical analysis?
I am trying to understand randomization in experiment design, and am very confused, because there appear to be several types of randomization possible.
For example, for a Categorical Factor with Non-...
0
votes
0answers
8 views
On regressing data to determine a material property: the finite intercept or the theoretical (zero-intercept) model approach?
Suppose I manufacture and sell simple springs. I advertise the spring constant $k$ and its uncertainty ($k\pm \delta k$) of my springs, where $k$ is related to the magnitude of the spring's restoring ...
2
votes
0answers
36 views
What does "nearly orthogonal" mean for a matrix?
I am currently working on experimental designs. I have read several times (e.g Kuhfeld 2010) that for the main effects to be identified, the design matrix has to be orthogonal or "nearly ...
0
votes
0answers
7 views
What is the best way to design a inter vs intra-operator reliability study?
I am hoping to get a good idea on how to create and write up a design to measure the variability and accuracy of an imaging technique involving transnational medical data.
Ideally - the study would ...
1
vote
0answers
15 views
How to specify a conservative Design of Experiment when runs could be missing?
I have to design an experiment with 3 input variables $x_1$, $x_2$, $x_3$ and I have 25 available runs which I can use. A full factorial design with center point and 2 replications would be 18 runs, ...
0
votes
0answers
12 views
Research design: Term for assigning participants to groups depending on day (instead of randomisation)
I am looking for the term used to describe when participants are assigned to experiment groups based on the day. We tested several classes in a training centre and selected randomly which class should ...
1
vote
0answers
21 views
Confusion on Ablation test (Ablation Experiment or Ablation study)
I followed the steps of the ablation test to calculate the feature importance one by one.
In Table 1, row 1 presents the model prediction performance of using full features. Regarding rows 2-4, these ...
1
vote
1answer
17 views
Is 2AFC a good way to determine the preferred choice?
We have two stimuli, A and B, for which we would like to determine if there is a significant preference among users. (For some background info: A and B are 3D scenes rendered with different ...
0
votes
0answers
24 views
Mixed Multilevel Model
I'm writing as I have hopelessly been looking for an answer to my problem and checking if someone can help me here. I know there are many multilevel questions around there, but they don't seem to ...
0
votes
0answers
29 views
Unable to use the parameter estimates from R output to get the mean of a group
I have built a statistical model to predict score/performance and have achieved the following parameter estimates:
Yi = 58.75 + 3.25(X1i) + 0.75(X2i) + (1.25X3i)
...
2
votes
1answer
81 views
Reconciling the Minimum Detectable Effect Calculation with the Experiment Result
I am puzzled why I am able to detect a statistically significant effect that is smaller than the MDE from the power calculation for the same set of parameters.
Here's Stata code for for the familiar ...
3
votes
1answer
39 views
Specifying a multilevel model: an experimental psychologists nightmare
Thank you greatly for your help in answering this question.
I am having some trouble figuring out how to define my multilevel model. Firstly, a description of what I have completed. This is the ...
0
votes
1answer
24 views
How to control for different data sources in each independent categorical Variable linear mixed model
I want to investigate, how gender (male/femlae) and body postures (open/closed) of physicians on pictures change the ratings of participatns (several likert scale questions)
Basically it is a 2*2 ...
0
votes
0answers
31 views
I'm looking for a statistical test more robust than the sign test for my experimental analysis
I made an exploratory experiment that consists of a pair of readings. I used Scanning Electron Microscopy-Energy Dispersive Spectroscopy (SEM-EDS) to read elements on several mineral surfaces, getting ...
0
votes
1answer
31 views
Different ANOVAs per species for same variable; how to proceed if only some species hold non-normal data
I am conducting two-way ANOVAs to determine if plant height (mm) is affected by treatment (3 water regimes) in different growth medium types (2). I am also interested in the interactions that may ...
0
votes
0answers
16 views
What is the filter experiment?
I read a method to check which model does best in measuring uncertainty.
Source: page 485 of Probabilistic Deep Learning: with Python, Keras and Tensorflow Probability
The steps are as follows:
...
1
vote
0answers
21 views
How to identify the effects of treatments relative to a control group?
I am currently designing a vignette survey experiment with three conditions. Respondents will either be assigned to (a) the first treatment, (b) the second treatment, or (c) the control group. After ...
4
votes
2answers
319 views
How to test if two means are significantly similar?
I'm trying to run an experiment to see if my test and control (in this case 2 survey types) are similar, as opposed to significantly different, from each other.
Is there a test/way to do this? almost ...
0
votes
0answers
11 views
Balanced Allocation of Subjects to Base/Target
Lets say I'm conducting an experiment / AB test on a very volatile measure.
I have a set of good estimators for this measure, which is known to me at the moment I need to assign each subject/...
0
votes
0answers
12 views
Factor analysis with different experimental conditions: Implications(?)
I have conducted (exploratory) factor analysis (EFA) on data from a within-subject design with four different experimental conditions. Each participant goes through all conditions one after the other (...
1
vote
0answers
25 views
Does the Fisher information matrix have to be convex when assessing an optimal criteria?
I've reached a local minimum for a proposed model using a set of experimental data (positive definite Hessian) and want to select an additional experiment that will reduce the parameter uncertainties (...
0
votes
0answers
40 views
To check the similarity of distribution of categorical variables in Treatment and Control group
I have a dataset with 19variables. The dataset has a column as GROUP and it has 2 values- Treatment and Control. There is a column as CONVERTED which has value 'Yes' 'No'. Basically, I want to ...
2
votes
2answers
50 views
Is it always better to analyze at the most granular level possible? What is the best unit of analysis in the context of hierarchically clustered data?
Let's say that I have experimental data where the level of treatment is at a higher level of aggregation than the level of observation. For example, imagine some subset of schools adopted a new ...
2
votes
0answers
38 views
Fisher information matrix for model based design of experiments
I'm trying to use the Fisher information matrix and D-optimal design for model-based experiment selection, but I'm not sure if I'm implementing or interpreting it correctly.
I begin with the values of ...