7
votes
Accepted
The probability of making a Type S error, and the average amount of magnification (type M error) as a function of power
Sketch of the t-test
Let $x_1, \dots, x_n \sim N(\mu_1,\sigma^2)$ and $y_1, \dots, y_n \sim N(\mu_2,\sigma^2)$ be independent samples. Let's define
The raw effect
$\theta = \mu_2-\mu_1$
The estimate ...
- 86.5k
7
votes
Accepted
Statistical testing on non-random sample?
The result of your analysis (regression, or whatever you did) are valid for your sample; the effect sizes are accurate estimates for your sample. But the validity of any generalization relies on your ...
- 128k
6
votes
Accepted
What is a "split-body" RCT?
It's a trial whether the two interventions are applied to different parts of the same person. One setting is ophthalmology, with different treatments applied to each eye, but this is an example with ...
- 46.8k
6
votes
Do I need to test for autocorrelation or normality assumption if I am running the regression with standard errors?
With so many observations, tests for normality or autocorrelation will most likely end up giving extremely low $p$-values, suggesting to reject the null.
Using robust standard errors is fine and ...
- 12.1k
6
votes
Accepted
Analyzing an experiment which consists of many "small" experiments
This is a standard A/B test if you view cakes as the sample from a population of products (rather than customers), with a varying number of taste tests (aka trials) for each cake.
Your proposed ...
- 38.3k
5
votes
Is most published research indeed false?
To have a sensible discussion about this, we first need to clarify what we mean by "false". The title of the 2005 paper is obviously a short hand, intended to attract the curiosity of the reader.
If ...
- 8,699
5
votes
Accepted
Study design question: What's the best design to assess harm of an exposure?
I start with my methodological thoughts and I offer some footnotes with thoughts that came to my mind on the ethics. Take both of these with a large grain of salt, because we know very little on your ...
- 35.2k
5
votes
Accepted
Model selection with low N?
For biomedical studies, a general rule of thumb to avoid overfitting in an unpenalized logistic regression model is to have on the order of 10-20 minority-class cases per evaluated predictor. You have ...
- 102k
5
votes
Latent Profile Analysis and Statistical Methods in Psychology
Latent Profile Analysis (LPA) is a term typically used for a model which identifies latent sub-populations within a population based on a certain set of categorical variables. In your case, you have ...
- 65.8k
5
votes
What does the phrase "adjusted by" mean?
"Adjusted by" or "conditioned on" is a phrase that can mean many different things, depending on the context. The New Causal Revolution, led by Judea Pearl and Donald Rubin, ...
- 6,039
5
votes
Do I need to test for autocorrelation or normality assumption if I am running the regression with standard errors?
As @utobi correctly notes in another answer, with such a large data set almost any test of a violation of model assumptions will tend to produce "statistically significant" results that ...
- 102k
5
votes
Relationship between Cohen's D effect size and p-value
A p-value won't inform you about the importance of the effect size (here Cohen's d). As for interpreting p-values, you may find the following discussions relevant: Understanding p-values using an ...
- 5,848
5
votes
Between two variables with weak correlations and no significant prediction rate from simple regression, what are the next research steps?
You could look into statistical interactions. A community with many low-income households may have higher burglary rates when there is a substantial number of high-income households as well. If you ...
- 12.9k
5
votes
Between two variables with weak correlations and no significant prediction rate from simple regression, what are the next research steps?
I'm not sure a typical regression works here. Since you are dealing with rates (and to my knowledge crime rates are discrete), this may be better modeled using a discrete response approach, perhaps ...
- 18.4k
4
votes
Accepted
Real noise modeling/ noise map generation (image processing, deep learning)
Let's say you have $m$ noisy images $I_N^{i};i \in [1...m ]$. One way of accomplishing your end goal is as follows:
Run single-image learning-based denoising techniques on your noisy images to obtain ...
- 605
4
votes
How to deal with treatment effect at state level but observations at school level?
I take it you're measuring at the school level and that
Schools are nested within states
States are either intervened upon or not
In what follows, I'll just assume the likelihoods are normal for ...
- 39.6k
4
votes
Study design question: What's the best design to assess harm of an exposure?
We have retrospective data suggesting worse survival for patients who received steroids, even when controlling for the indication for the steroids. Therefore it's not ethical to randomize patients to ...
- 64.8k
4
votes
Accepted
Advantages/disadvantages of fractional factorial design vs completely randomized design
What you propose is to randomize the choice of treatments. That is not a completely randomized design (CRD), a CRD is about randomizing the allocation of experimental units to the treatments.
Good ...
- 82.8k
4
votes
Study design question: What's the best design to assess harm of an exposure?
The scholarly literature guiding you into the best way to test a scientific hypothesis of comparative clinical effectiveness is quite large.
Have a look for instance at Rosenbaum's Design of ...
- 2,622
4
votes
Accepted
What does the phrase "adjusted by" mean?
I will presume, that in your example, Factor is the treatment, BodyMassIndex is the outcome, and you are interested in the ...
- 11.2k
4
votes
The probability of making a Type S error, and the average amount of magnification (type M error) as a function of power
To avoid notational difficulties, I will use notation to Gelman and Carlin, with the effect size represented as $\theta$ and upper-case $D$, $D^\text{rep}$, etc., used to represent the data as a ...
- 133k
4
votes
IVs: would this rainfall specification cut it?
You'd probably need more context knowledge for a more comprehensive answer, but let me try to give a brief evaluation of using rainfall in $B$ as IV for industrial activity $\to$ mortality in $A$ ...
- 2,439
4
votes
Is there a good reason for a lab to repeat experiments instead of conducting a single larger blocked experiment
Why might it be better to plan to repeat experiments within your lab (each with its own analysis) instead of designing and analysing a single larger experiment properly? And if repeating experiments ...
- 102k
4
votes
Statistical testing on non-random sample?
I come to this from a very different angle than @JohnX. For me the first thing to acknowledge here is that statistical models are thought constructs. There is an essential difference between formal ...
- 28.3k
4
votes
Three continous variables + 2 factors vs. five continous variables to control for confounders?
The effective sample size at your disposal for estimating model parameters will depend on the actual distribution of the 0-30 dependent variable, especially where there are a lot of ties creating a ...
- 98.4k
4
votes
Accepted
Logistic regression and study design with multiple groups
Your problem here is that your outcome of interest is not actually binary. There are three possible outcomes (none, mild, severe). What you are effectively doing by running multiple binary logit ...
- 6,020
3
votes
Latent Profile Analysis and Statistical Methods in Psychology
This question touches on both conceptual and statistical points, and I'd advise people in this situation to start with the conceptual part first.
There has been a popular phenomenon that has several ‘...
- 1,528
3
votes
Accepted
Imputing missing data using testing data
The out-of-sample data mimic the real situation of applying the model to unseen data, such as expecting Siri or Alexa to understand speech that has yet to be uttered, perhaps even by people who have ...
- 67.1k
3
votes
Accepted
Why do discretised predictors have lower statistical power than continuous predictors?
Smaller Variance Leads To Smaller Power
Discretizing predictors is a bad idea as Frank Harrell explains here. As to your titular question, let's examine a simple scenario.
Let's say I have a ...
- 39.6k
3
votes
What are several examples of survey questions which have response options designed as mutually exclusive when they are not?
A common example is discrete counts with overlapping class-limits. For example: How many days did you visit Stack Exchange in the last month?
0
1
2-5
5-10
more than 10
The problem is this...which ...
Only top scored, non community-wiki answers of a minimum length are eligible