Skip to main content
7 votes
Accepted

The probability of making a Type S error, and the average amount of magnification (type M error) as a function of power

Sketch of the t-test Let $x_1, \dots, x_n \sim N(\mu_1,\sigma^2)$ and $y_1, \dots, y_n \sim N(\mu_2,\sigma^2)$ be independent samples. Let's define The raw effect $\theta = \mu_2-\mu_1$ The estimate ...
Sextus Empiricus's user avatar
6 votes

Do I need to test for autocorrelation or normality assumption if I am running the regression with standard errors?

With so many observations, tests for normality or autocorrelation will most likely end up giving extremely low $p$-values, suggesting to reject the null. Using robust standard errors is fine and ...
utobi's user avatar
  • 11.8k
5 votes

Is most published research indeed false?

To have a sensible discussion about this, we first need to clarify what we mean by "false". The title of the 2005 paper is obviously a short hand, intended to attract the curiosity of the reader. If ...
Florian Hartig's user avatar
5 votes
Accepted

Model selection with low N?

For biomedical studies, a general rule of thumb to avoid overfitting in an unpenalized logistic regression model is to have on the order of 10-20 minority-class cases per evaluated predictor. You have ...
EdM's user avatar
  • 92.5k
5 votes

Latent Profile Analysis and Statistical Methods in Psychology

Latent Profile Analysis (LPA) is a term typically used for a model which identifies latent sub-populations within a population based on a certain set of categorical variables. In your case, you have ...
Robert Long's user avatar
  • 60.9k
5 votes
Accepted

What is a "split-body" RCT?

It's a trial whether the two interventions are applied to different parts of the same person. One setting is ophthalmology, with different treatments applied to each eye, but this is an example with ...
Thomas Lumley's user avatar
5 votes

What does the phrase "adjusted by" mean?

"Adjusted by" or "conditioned on" is a phrase that can mean many different things, depending on the context. The New Causal Revolution, led by Judea Pearl and Donald Rubin, ...
Adrian Keister's user avatar
5 votes

Do I need to test for autocorrelation or normality assumption if I am running the regression with standard errors?

As @utobi correctly notes in another answer, with such a large data set almost any test of a violation of model assumptions will tend to produce "statistically significant" results that ...
EdM's user avatar
  • 92.5k
5 votes

Relationship between Cohen's D effect size and p-value

A p-value won't inform you about the importance of the effect size (here Cohen's d). As for interpreting p-values, you may find the following discussions relevant: Understanding p-values using an ...
J-J-J's user avatar
  • 4,243
4 votes
Accepted

Real noise modeling/ noise map generation (image processing, deep learning)

Let's say you have $m$ noisy images $I_N^{i};i \in [1...m ]$. One way of accomplishing your end goal is as follows: Run single-image learning-based denoising techniques on your noisy images to obtain ...
PAF's user avatar
  • 605
4 votes
Accepted

Advantages/disadvantages of fractional factorial design vs completely randomized design

What you propose is to randomize the choice of treatments. That is not a completely randomized design (CRD), a CRD is about randomizing the allocation of experimental units to the treatments. Good ...
kjetil b halvorsen's user avatar
4 votes

How to deal with treatment effect at state level but observations at school level?

I take it you're measuring at the school level and that Schools are nested within states States are either intervened upon or not In what follows, I'll just assume the likelihoods are normal for ...
Demetri Pananos's user avatar
4 votes
Accepted

What does the phrase "adjusted by" mean?

I will presume, that in your example, Factor is the treatment, BodyMassIndex is the outcome, and you are interested in the ...
frank's user avatar
  • 10.8k
4 votes

The probability of making a Type S error, and the average amount of magnification (type M error) as a function of power

To avoid notational difficulties, I will use notation to Gelman and Carlin, with the effect size represented as $\theta$ and upper-case $D$, $D^\text{rep}$, etc., used to represent the data as a ...
Ben's user avatar
  • 125k
4 votes

Can you compare Propensity Score Matching estimates?

I am not positive, but I think you could do this from a purely statistical perspective. The difference between getting and not getting a scholarship will be estimated for each equation and those two ...
Peter Flom's user avatar
  • 120k
4 votes

Can you compare Propensity Score Matching estimates?

Propensity score matching is an inefficient way to adjust for confounding. It is not needed unless the number of potential confounders is so large that you can’t model them as individual covariates. ...
Frank Harrell's user avatar
4 votes

Is there a good reason for a lab to repeat experiments instead of conducting a single larger blocked experiment

Why might it be better to plan to repeat experiments within your lab (each with its own analysis) instead of designing and analysing a single larger experiment properly? And if repeating experiments ...
EdM's user avatar
  • 92.5k
4 votes
Accepted

Statistical testing on non-random sample?

The result of your analysis (regression, or whatever you did) are valid for your sample; the effect sizes are accurate estimates for your sample. But the validity of any generalization relies on your ...
Peter Flom's user avatar
  • 120k
3 votes
Accepted

Imputing missing data using testing data

The out-of-sample data mimic the real situation of applying the model to unseen data, such as expecting Siri or Alexa to understand speech that has yet to be uttered, perhaps even by people who have ...
Dave's user avatar
  • 62.6k
3 votes

What are several examples of survey questions which have response options designed as mutually exclusive when they are not?

A common example is discrete counts with overlapping class-limits. For example: How many days did you visit Stack Exchange in the last month? 0 1 2-5 5-10 more than 10 The problem is this...which ...
3 votes

IVs: would this rainfall specification cut it?

You'd probably need more context knowledge for a more comprehensive answer, but let me try to give a brief evaluation of using rainfall in $B$ as IV for industrial activity $\to$ mortality in $A$ ...
Scriddie's user avatar
  • 2,289
3 votes

Three continous variables + 2 factors vs. five continous variables to control for confounders?

The effective sample size at your disposal for estimating model parameters will depend on the actual distribution of the 0-30 dependent variable, especially where there are a lot of ties creating a ...
Frank Harrell's user avatar
2 votes

Regression with mostly binary and categorical variables in R

You can use the family=binomial option of the glm() function, and specify the # correct answers and # incorrect answers as the ...
mastropi's user avatar
  • 183
2 votes

Is there a difference between controlling for population size directly vs. putting variables in per capita terms?

I answered your more general question, Controlling for population size, using per capita or including a variable for population size with reference to this one. To address this more specific question: ...
AJV's user avatar
  • 687
2 votes

Is there a difference between controlling for population size directly vs. putting variables in per capita terms?

Its not that simple, at least not in a linear additive model. It also fundamentally depends on your dependent variable. Take a simple example where the true Data Generating Process (DGP) a increase ...
Mark Verhagen's user avatar
2 votes

Crossing Frequentism and Bayesian Analysis

I would say that is indeed done a lot. That said, it is correct that this is done less often in Bayesian work, since the focus tends to be more on finding a good answer given the data at hand, where ...
Christoph Hanck's user avatar
2 votes

Evaluate change in participant population characteristics

If you have an established cohort, and assuming the demographic covariates of interest between the time points are static, why not try a multi-class classifier? In this case, you would have four ...
dmartin's user avatar
  • 3,315
2 votes

How to do meta-analysis of diagnostic tests with only sensitivity and specificity?

Comment (too long for 'comment' format): When dealing with 'prevalence', 'specificity', 'specificity', and so on. It is important to be clear to what population each probability applies. Prevalence is ...
BruceET's user avatar
  • 56.4k
2 votes

Latent Profile Analysis and Statistical Methods in Psychology

This question touches on both conceptual and statistical points, and I'd advise people in this situation to start with the conceptual part first. There has been a popular phenomenon that has several ‘...
Weiwen Ng's user avatar
  • 1,428
2 votes

In a RCT, what are the risk of selection bias if no allocation bias is present?

If you merely assume away the problem, then obviously that is going to make it difficult to see the dangers of that kind of design. Unfortunately, reality does not let us assume away statistical ...
Ben's user avatar
  • 125k

Only top scored, non community-wiki answers of a minimum length are eligible