# Questions tagged [regression-strategies]

Regression Modeling Strategies

289 questions
Filter by
Sorted by
Tagged with
1k views

### How to start building a regression model when the most strongly associated predictor is binary

I have data set containing 365 observation of three variables namely pm, temp and rain. Now ...
1 vote
50 views

### Use R to perform a logistic regression and cross-validate in a small sample?

I have a small sample (n=69, 35 with diseases, 34 without disease). I have around 20 variables that came up as significant using indiscriminate univariate regression. Of these 20 variables, several ...
• 11
159 views

### Estimation of treatment effect when there is an unknown and variable coverage of the population

I am not sure if I am using the correct terminology, something must be written about the following problem, but I cannot find it by searching. I am presently analyzing data about the effect of ...
• 64.7k
1 vote
997 views

### Why does gvlma() give contradictory results for linear model assumptions?

I have fitted a linear model and I am checking the assumptions for which I get this diagnostic plot: From this diagnostic plot there seems to be an increasing variance in the residuals for increasing ...
• 93
369 views

### Do the problems of stepwise variable selection exist in FA, PCA, SEM?

Note: This is a revision of my original question. I have read the critique of stepwise variable selection and "all possible subsets regression" by Professor Frank Harrell here. Are factor analysis, ...
• 96
1 vote
84 views

### Discovering transformations and interactions

I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the ...
• 13
32 views

### Discovering interactions and transformations [duplicate]

I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the ...
• 13
43 views

### Is having more features definitely equal to having a higher chance of overfitting?

I am doing a EEG data classification problem. Currently I am using the ANOVA test to help me select K best input features (with K a parameter to tune) and feeding the selected features into a logistic ...
• 203
2k views

### Alternative to linear regression

I need to run hundreds of linear regression models, with the same set of independent variables, but with varying dependent variables. I have checked normality for a few dozens. Some are normally ...
• 685
7k views

### Goodness of fit, predictive power, discrimination

I'm making a couple of logistic regression based predictive models and intend to compare them and see which is "best". "best" here is obviously ill-defined, but as I'm looking for common metrics for ...
• 843
419 views

### Problems with zero values when testing for linearity of the logit

From Field's 'Discovering Statistics using SPSS' I read that you should test for linearity of the logit when using a logistic regression. The process explained in the book covers the natural log ...
• 163
1 vote
2k views

### Logistic Regression Cutoff Values for Multiple Models

I understand that once the logistic regression model has output probabilities, a cutoff value for classifying probabilities of new observations is decided for a model to optimize some metric like ...
• 71
8k views

### Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
• 111
174 views

### Does the p value for logistic regression depends on odds ratio or logit?

The p value in a logistic regression is given for the B estimates. But what if I'm reporting odds ratio instead of logit. Do the same p values apply?
• 61
1k views

### R: Quadratic Regression with interaction: when to center?

I have a statistical question. I have data from an experiment with two conditions (dichotomous IV: 'condition'). I also want to make use of another IV which is metric ('hh'). My DV is also metric ('...
• 41
162 views

### Optimal Cutpoint for Predicted Results from Kaplan Meier and Cox Regression

Is there anyway to get the optimal cutpoint for predicted survival probabilities of the aforementioned survival analysis approaches? Something like the optimal cutpoint from ...
• 15
93 views

### Is it normal for logistic regression, to have predictors which have good Wald's Chi Sq, but still bad performance?

I have trying to build a logistic model with some 10 variables. All of those variables have Wald Chi sq value<500. All are highly significant p-value<0.0001. Event rate is about 3%. The ...
• 717
1k views

### predict from a Cox model with beta coefficients

I fitted a Cox PH model in R with the survival package and the coxph function. I get the beta estimates from this model. How can I use these coefficients to ...
• 1,333
330 views

### Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in "rms"...
• 21
8k views

### Why does propensity score matching work for causal inference?

Propensity score matching is used for make causal inferences in observational studies (see the Rosenbaum / Rubin paper). What's the simple intuition behind why it works? In other words, why if we ...
• 1,254
1 vote
1k views

### How to use aregImpute "group" argument?

Can someone provide an example of using the "group" argument with aregImpute()? I see that group=NULL is the default, but my data include a few factor variables with levels with <5 observations. My ...
5k views

### How to make calibration plot for survival data without binning data?

To make a calibration plot for survival probabilities estimated from a Cox model, one can divide the estimated risk into groups, calculate the average risk within a group, and then compare this to the ...
• 2,262
3k views

### Logistic Regression on Time-dependent Predictors

I would like to know if I can apply the techniques, like say Logistic Regression, to data whose variables/predictors are 'indexed' by time. Or if not, what techniques are appropriate to use in these ...
1k views

### Why can't we use linear regression for discrete Y?

This text is from An Introduction to Statistical Learning with Applications in R(by • Gareth James • Daniela Witten • Trevor Hastie • Robert Tibshirani) List item Can anyone help me by making linear ...
• 239
1 vote
580 views

### linearity test for cublic splines

Consider two models in which a continuous variable is modeled as a restricted cubic spline (RCS) or entered linearly. If one carries out a test for linearity, why are the degrees of freedom for the ...
• 2,262
48 views

### What should be the ratio between number of cases and attributes in multivariate regression?

Is there any way to determine if it is feasible to perform a multivariate regression based on a given number of samples and attributes? For example I have a data set with 6 cases , 30 attributes and ...
• 11
5k views

### To select variables or not in logistic regression

I am trying to find predictors for an outcome. I was taught to perform univariate analyses & put significant variables into a multivariate logistic regression model. Then I remove variables one by ...
• 111
1 vote
92 views

### Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
• 113
309 views

### To use a two-sample t-test or to use a Mann-Whitney U-test on spatial data?

I have obtained some mean summer temperature records (June 1st- September 30th) for all years within the 1930s (1930-1939) and I have obtained summer temperature records for all years within the ...
• 83
1k views

### Randomizing Class Labels during classification to asses the feature selection results

I have a binary classification problem with thousands of variables and less than a hundred data points and class labels. The class is imbalanced (24 positive 51 negative samples). I have selected some ...
• 744
1 vote
1k views

### Which balance strategy to learn from my very imbalanced dataset?

I'm using a deep learning approach on a dataset made of ~20 millions of elements, where each element has a TRUE or FALSE label. This dataset unfortunately is veeery imbalanced: I've 98% of falses and ...
173 views

### What's wrong with data-guided modeling in regression?

In the Regression Modelling Strategies of Frank Harrell, section 4.1, if I understood correctly, it is not recommendred to using the data to decide how to represent a predictor in a regression model (...
• 2,376
6k views

### Minimum number of observations needed for penalized regression?

I'm wondering what should be the minimum sample size to perform ridge, lasso or elastic net regression. I have a binomial outcome that I want to relate with a set of features (18 features in some ...
9k views

### In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
• 121
439 views

### Logistic Regression with (Normal) Distributions for Independent Variables

Consider the logistic regression where $Y_i \in {0,1}$ are dependent variable observations and $X_i \in \mathbb{R}$ are the independent variables. However we do not observe the $X_i$ themselves. ...
• 622
7k views

### When is logistic regression suitable?

I'm currently teaching myself how to do classification, and specifically I'm looking at three methods: support vector machines, neural networks, and logistic regression. What I am trying to understand ...
• 6,019
2k views

### Can we correctly identify all the non-zero coefficients in the linear regression model?

I have a conceptual question regarding linear regression. Assume our model is correct, i.e., the response variable $Y$ is indeed coming from the model $$Y=\beta_0+\beta X+\epsilon.$$ Here $X$ is ...
• 6,407
168 views

### Hosmer–Lemeshow test - Best Model

I have 3 different models and I do the Hosmer–Lemeshow test. I have a p-value and a Chi2 value. How can I know which model fits the best my data? Khi-2 || Pr > Khi-2 12.04 || 0.19 7.47 || 0.71 4....
2k views

### How to find marginal effect of restricted cubic spline

I'm trying to figure out how to find the marginal effect of an interaction term from a restricted cubic spline in a non-linear model. The post Nonlinear effect in an interaction term is a good start ...
• 241
20k views

### Ranking features in logistic regression

I used Logistic Regression. I have six features, I want to know the important features in this classifier that influence the result more than other features. I used Information Gain but it seems that ...
• 211
1k views

### How to apply a model on dataset with missing data?

This question is similar to Missing input value during prediction of a generalized linear model. Consider the following scenario: I fitted a linear regression model on a training dataset with ...
• 581
56 views

### How to choose between data-driven pattern or intuition?

I am performing a multivariate logistic regression (This could very well be any other kind of regression method) to study the effect of some predictor variables on the probability of event. I have ...
• 581
3k views

### What to do with non-normality and heterogeneous variances in two-way ANOVA when transformations do not work?

I'm conducting a Two-Way ANOVA with my two factors being Sex and Cohort. I have data from two cohorts of subjects, with each cohort consisting of males and females that were measured on one response ...
• 41
300 views

### Is my understanding of regularized logistic regression correct?

I learned that regularized logistic regression helps prevent the model from over-fitting the data. I understand that the function is still technically a high-order polynomial, but the effect is ...
1 vote
408 views

### Using a dichotomous categorical variable that has an underlying continuous dichotomy in a multiple regression?

FINAL EDIT I just found a good answer to this question in another thread in the forum here, therefore, I think this question could be closed. Thanks so much for your help and the clarification about ...
• 11
1 vote
190 views

### logistic regression predictive modeling

I would like to use a logistic regression for estimating the parameters of the logit function by using the maximum likelihood estimate. This amounts to minimizing the log-loss function, instead of ...
• 25
1 vote
2k views

### Test/Measure for Rank Ordering a Logistic Regression model, invariant to event rate and population size

I have a model whose purpose is to rank order event risk - the output of which is split into twentiles (which have been based off the benchmark data). Currently, I'm using Somers' D calculated on ...
578 views

### How to best model interaction effect of two continuous predictor variables?

Consider the following problem: In a logistic regression model, we believe that two continuous predictor variables $X_1$ and $X_2$ impact the probability of event. It is hypothesized that the ...
• 581
1 vote