Background: This cross-sectional study collected 30 thrombosis samples. We evaluated the presence or absence of MP components (dependent variable), where 24 cases had MP (coded as 1) and 6 cases did not (coded as 0). We performed single- and multi-factor logistic regression analyses, using clinical indicators as independent variables and the presence or absence of MP as the dependent variable.
0
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
5
I do not think the problem comes from the small sample size per. It rather comes from the fact that MPs are obviously almost present everywhere regardless from the value of explanatory variables, since only 6 samples out of 30 have a score of 0, and all the rest are 1s. There is barely any variance to explain. Therefore, including more explanatory variables seems irrelevant here.
I would try to obtain a continuous measure of MP instead. If this is not possible, I would not run any statistics, but simply say that MP can be found in most samples regardless of the value of the explanatory variables.
-
$\begingroup$ This sounds more like a comment than an answer. $\endgroup$ Commented Feb 27 at 11:51
-
1$\begingroup$ The concentration of MP is also one of the outcome variables, which is a continuous variable. Therefore, we have already used a multiple linear regression model for statistical analysis. $\endgroup$ Commented Feb 27 at 11:59
-
$\begingroup$ So then why do you want to analyse it as presence/absence when it is obviously almost always present ? $\endgroup$– CaroZCommented Feb 27 at 12:04
-
$\begingroup$ Because I am a beginner in statistics, and my methods are based on referencing papers from similar researchers, so there may be many errors. I am curious, is it unreasonable to analyze the presence of MP based on the assumption of a high probability of time occurrence (80%)? What are the specific reasons for this? $\endgroup$ Commented Feb 27 at 12:13
-
$\begingroup$ What is it you call "time occurrence" ? For the mathematical reason, see Frank Harrell's answer above. I can only tell you an answer based on common sense : if something is present almost everywhere, you do not have enough cases of non-occurrence to estimate the probability of non-occurrence. It would be a case of really wanting to fit an analysis in somewhere where a simple description of what you observe would be accurate. $\endgroup$– CaroZCommented Feb 28 at 8:35
$\begingroup$
$\endgroup$
4
There is no fix for this situation. Not only has the experiment used a minimum information (binary) response variable, but it does not have the needed sample size (96) just to estimate the intercept in a logistic regression model. To add to the problem the distribution of Y is not balanced. The sample size needed here is more like 96 + 15 times the number of candidate predictors. The effective sample size is 3np(1-p) where p is the proportion of positive responses. More in https://hbiostat.org/rmsc/multivar and https://hbiostat.org/rmsc/lrm . The referee is exactly correct.
answered Feb 27 at 12:50
-
$\begingroup$ Is the sample size needed (96 + 15k) about the effective sample size in binary regression? For example, n = 200, p = 0.5, then effective sample size is 150, which supports estimation of one intercept and three slopes. The raw sample size 200 seems to support one intercept and seven slopes instead. $\endgroup$ Commented Mar 13 at 23:21
-
$\begingroup$ A much better formula is in onlinelibrary.wiley.com/doi/10.1002/sim.7992 which shows that rules of thumb aren't super reliable. But the 96 is very solid as the minimum. $\endgroup$ Commented Mar 14 at 22:36
-
$\begingroup$ Thanks. The paper argues that event per parameter (EPP) is not a constant threshold but the global shrinkage factor at 0.9 is. I am curious why the resulting EPP is not symmetrical around event definition in binary logit models. In Section 2.4, if the outcome proportion is 0.1, the calculated EPP is 8.5. Accordingly, EPP = (1698 × 0.9)/20= 76.4 is the outcome proportion is 0.9. But if we flip the outcome definition to its negation, so outcome proportion 0.9 is equivalent to negation proportion 0.1, the EPP should be still 8.5, contrasting to the much higher 76.4 suggested in the paper. $\endgroup$ Commented Mar 15 at 11:55
-
$\begingroup$ First of all, EPP is not a great concept so don't put that much stock in it. Second, it is based on the lower of the two proportions p, 1-p. Think of the effective sample size for binary Y which is symmetric: 3np(1-p). The effective sample size for continuous Y is n. $\endgroup$ Commented Mar 15 at 13:17
$\begingroup$
$\endgroup$
The usual rule of thumb with logistic regression is that you can include one IV for every 10 cases in the less common case. For you, that's six people, so this rule would say "One IV at most".
The problem here is not your method, but the data. You haven't got enough. Overfitting happens when you try to fit models to data like this.
From my small amount of research, modified Poisson is a robust method, intended to deal with problems such as clustered data. That's not your issue. Exact logistic may help, but I think only by making your results less significant and your CIs wider. This may not seem like help, but it is, in that, when you have overfitting, your p values are likely to be too small and your CIs too narrow. See King and Ryan (2002) A Preliminary Investigation of Maximum Likelihood Regression vs. Exact Logistic Regression, particularly the section on the ESR data.
I am not a Bayesian, but my guess on that is that the results of a regression with so little data ought to have very little effect on your priors (whatever they are).
$\begingroup$
$\endgroup$
- Can we still use a multi-factor logistic regression model? 2. The results of the single-factor logistic regression analysis suggest that the D-dimer concentration index has statistical significance (P=0.034) in relation to the presence or absence of MP, while the other indicators have p-values greater than 0.05. Can we include variables such as gender and age in the multi-factor logistic regression model? How many variables should be included at most in the multi-factor logistic regression model?
As others have pointed out, the sample size is too small to allow many predictors. Too many predictors of a binary outcome tend to results in complete separation with large coefficients and huge standard errors. Coefficients in generalized linear models are asymptotically consistent, meaning that they are biased but the biases get smaller when the sample size increases. Similarly, their standard errors are at best consistent. In small samples, this bias may be relatively large. The p values you reported whether based on Wald, score, or likelihood-ratio tests are not trustworthy because they require asymptotic interpretations, which only apply when the sample size is large enough. See my comment on these tests Likelihood ratio vs. score vs. Wald test: Different p values, which to use?.
- If the multi-factor logistic regression model is not feasible, can we only perform a single-factor logistic regression analysis to demonstrate the influence of D-dimer on the dependent variable?
Not a good idea to abandon multiple predictors if they actually affect the outcome and are correlated among each other. This will result in omitted variables, which bias the estimate.
- If logistic regression is not used, can we utilize the adjusted Poisson regression (robust Poisson regression) mentioned in the referenced article (https://mp.weixin.qq.com/s/1EQjsKvLyWXRVFMX_u0y-w) and website (https://mengte.online/archives/11695)? Can the 95% CI of the odds ratio converge to single-digit values?
No, this will not solve the small-sample size problem. Poisson regression directly reports the relative risk estimate. But one can easily get the same quantity in logistic regression by using post-estimation tools. See marginaleffects::comparisons() in R. Here the "robust" is about the standard error only, which tries to correct the coefficient variance-covariance matrix by a sandwich estimator to make it consistent. "Consistent" means that it approaches the true value as sample size increases. With the sample size fixed and small, the robustness or consistency of standard errors are not very insightful.
- If none of the above approaches are suitable, what other statistical methods can be employed? Bayesian function？Exact logistic regression.
The exact logistic regression appears to be the best option. It compares to permutation test in one-factor analysis. See a tutorial https://stats.oarc.ucla.edu/r/dae/exact-logistic-regression/ which also has a sample size of 30. "Exact logistic regression is used...when the sample size is too small for a regular logistic regression."
The concentration of MP is also one of the outcome variables, which is a continuous variable. Therefore, we have already used a multiple linear regression model for statistical analysis.
You have a sample-selection issue in the linear regression. We can only observe MP concentration if MP is present. Therefore, the effect of a predictor on MP concentration is biased if we estimate it from the 24 observations where MP is present. See Toomet, O., & Henningsen, A. (2008). Sample selection models in R: Package sampleSelection. Journal of Statistical Software, 27(7). https://doi.org/10.18637/jss.v027.i07 and Toomet, O. (2020). Treatment effects with normal disturbances in sampleSelection package. https://cran.r-project.org/web/packages/sampleSelection/vignettes/treatReg.pdf. I believe these sample-selection correction operates also on large-sample properties. Nevertheless, you can bootstrap the small sample and get exact p values. I guess we need stratified bootstrap here, each time sampling six cases from the non-MP cases and 24 cases from the MP cases with replacement.
