Questions tagged [regression-strategies]

Regression Modeling Strategies

Filter by
Sorted by
Tagged with
11 votes
2 answers
1k views

How to start building a regression model when the most strongly associated predictor is binary

I have data set containing 365 observation of three variables namely pm, temp and rain. Now ...
user avatar
1 vote
0 answers
50 views

Use R to perform a logistic regression and cross-validate in a small sample?

I have a small sample (n=69, 35 with diseases, 34 without disease). I have around 20 variables that came up as significant using indiscriminate univariate regression. Of these 20 variables, several ...
user avatar
  • 11
2 votes
1 answer
159 views

Estimation of treatment effect when there is an unknown and variable coverage of the population

I am not sure if I am using the correct terminology, something must be written about the following problem, but I cannot find it by searching. I am presently analyzing data about the effect of ...
user avatar
1 vote
0 answers
997 views

Why does gvlma() give contradictory results for linear model assumptions?

I have fitted a linear model and I am checking the assumptions for which I get this diagnostic plot: From this diagnostic plot there seems to be an increasing variance in the residuals for increasing ...
user avatar
  • 93
2 votes
0 answers
369 views

Do the problems of stepwise variable selection exist in FA, PCA, SEM?

Note: This is a revision of my original question. I have read the critique of stepwise variable selection and "all possible subsets regression" by Professor Frank Harrell here. Are factor analysis, ...
user avatar
  • 96
1 vote
1 answer
84 views

Discovering transformations and interactions

I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the ...
user avatar
  • 13
0 votes
0 answers
32 views

Discovering interactions and transformations [duplicate]

I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the ...
user avatar
  • 13
2 votes
0 answers
43 views

Is having more features definitely equal to having a higher chance of overfitting?

I am doing a EEG data classification problem. Currently I am using the ANOVA test to help me select K best input features (with K a parameter to tune) and feeding the selected features into a logistic ...
user avatar
2 votes
1 answer
2k views

Alternative to linear regression

I need to run hundreds of linear regression models, with the same set of independent variables, but with varying dependent variables. I have checked normality for a few dozens. Some are normally ...
user avatar
8 votes
1 answer
7k views

Goodness of fit, predictive power, discrimination

I'm making a couple of logistic regression based predictive models and intend to compare them and see which is "best". "best" here is obviously ill-defined, but as I'm looking for common metrics for ...
user avatar
  • 843
2 votes
1 answer
419 views

Problems with zero values when testing for linearity of the logit

From Field's 'Discovering Statistics using SPSS' I read that you should test for linearity of the logit when using a logistic regression. The process explained in the book covers the natural log ...
user avatar
  • 163
1 vote
3 answers
2k views

Logistic Regression Cutoff Values for Multiple Models

I understand that once the logistic regression model has output probabilities, a cutoff value for classifying probabilities of new observations is decided for a model to optimize some metric like ...
user avatar
11 votes
2 answers
8k views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
user avatar
  • 111
0 votes
1 answer
174 views

Does the p value for logistic regression depends on odds ratio or logit?

The p value in a logistic regression is given for the B estimates. But what if I'm reporting odds ratio instead of logit. Do the same p values apply?
user avatar
  • 61
3 votes
0 answers
1k views

R: Quadratic Regression with interaction: when to center?

I have a statistical question. I have data from an experiment with two conditions (dichotomous IV: 'condition'). I also want to make use of another IV which is metric ('hh'). My DV is also metric ('...
user avatar
  • 41
0 votes
1 answer
162 views

Optimal Cutpoint for Predicted Results from Kaplan Meier and Cox Regression

Is there anyway to get the optimal cutpoint for predicted survival probabilities of the aforementioned survival analysis approaches? Something like the optimal cutpoint from ...
user avatar
  • 15
2 votes
1 answer
93 views

Is it normal for logistic regression, to have predictors which have good Wald's Chi Sq, but still bad performance?

I have trying to build a logistic model with some 10 variables. All of those variables have Wald Chi sq value<500. All are highly significant p-value<0.0001. Event rate is about 3%. The ...
user avatar
2 votes
2 answers
1k views

predict from a Cox model with beta coefficients

I fitted a Cox PH model in R with the survival package and the coxph function. I get the beta estimates from this model. How can I use these coefficients to ...
user avatar
  • 1,333
0 votes
1 answer
330 views

Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in "rms"...
user avatar
  • 21
17 votes
4 answers
8k views

Why does propensity score matching work for causal inference?

Propensity score matching is used for make causal inferences in observational studies (see the Rosenbaum / Rubin paper). What's the simple intuition behind why it works? In other words, why if we ...
user avatar
  • 1,254
1 vote
0 answers
1k views

How to use aregImpute "group" argument?

Can someone provide an example of using the "group" argument with aregImpute()? I see that group=NULL is the default, but my data include a few factor variables with levels with <5 observations. My ...
user avatar
9 votes
1 answer
5k views

How to make calibration plot for survival data without binning data?

To make a calibration plot for survival probabilities estimated from a Cox model, one can divide the estimated risk into groups, calculate the average risk within a group, and then compare this to the ...
user avatar
  • 2,262
0 votes
1 answer
3k views

Logistic Regression on Time-dependent Predictors

I would like to know if I can apply the techniques, like say Logistic Regression, to data whose variables/predictors are 'indexed' by time. Or if not, what techniques are appropriate to use in these ...
user avatar
  • 1
0 votes
1 answer
1k views

Why can't we use linear regression for discrete Y?

This text is from An Introduction to Statistical Learning with Applications in R(by • Gareth James • Daniela Witten • Trevor Hastie • Robert Tibshirani) List item Can anyone help me by making linear ...
user avatar
1 vote
1 answer
580 views

linearity test for cublic splines

Consider two models in which a continuous variable is modeled as a restricted cubic spline (RCS) or entered linearly. If one carries out a test for linearity, why are the degrees of freedom for the ...
user avatar
  • 2,262
0 votes
1 answer
48 views

What should be the ratio between number of cases and attributes in multivariate regression?

Is there any way to determine if it is feasible to perform a multivariate regression based on a given number of samples and attributes? For example I have a data set with 6 cases , 30 attributes and ...
user avatar
  • 11
7 votes
1 answer
5k views

To select variables or not in logistic regression

I am trying to find predictors for an outcome. I was taught to perform univariate analyses & put significant variables into a multivariate logistic regression model. Then I remove variables one by ...
user avatar
  • 111
1 vote
1 answer
92 views

Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
user avatar
  • 113
3 votes
1 answer
309 views

To use a two-sample t-test or to use a Mann-Whitney U-test on spatial data?

I have obtained some mean summer temperature records (June 1st- September 30th) for all years within the 1930s (1930-1939) and I have obtained summer temperature records for all years within the ...
user avatar
  • 83
3 votes
1 answer
1k views

Randomizing Class Labels during classification to asses the feature selection results

I have a binary classification problem with thousands of variables and less than a hundred data points and class labels. The class is imbalanced (24 positive 51 negative samples). I have selected some ...
user avatar
1 vote
2 answers
1k views

Which balance strategy to learn from my very imbalanced dataset?

I'm using a deep learning approach on a dataset made of ~20 millions of elements, where each element has a TRUE or FALSE label. This dataset unfortunately is veeery imbalanced: I've 98% of falses and ...
user avatar
7 votes
2 answers
173 views

What's wrong with data-guided modeling in regression?

In the Regression Modelling Strategies of Frank Harrell, section 4.1, if I understood correctly, it is not recommendred to using the data to decide how to represent a predictor in a regression model (...
user avatar
  • 2,376
2 votes
2 answers
6k views

Minimum number of observations needed for penalized regression?

I'm wondering what should be the minimum sample size to perform ridge, lasso or elastic net regression. I have a binomial outcome that I want to relate with a set of features (18 features in some ...
user avatar
12 votes
3 answers
9k views

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
user avatar
  • 121
8 votes
2 answers
439 views

Logistic Regression with (Normal) Distributions for Independent Variables

Consider the logistic regression where $Y_i \in {0,1}$ are dependent variable observations and $X_i \in \mathbb{R}$ are the independent variables. However we do not observe the $X_i$ themselves. ...
user avatar
  • 622
13 votes
2 answers
7k views

When is logistic regression suitable?

I'm currently teaching myself how to do classification, and specifically I'm looking at three methods: support vector machines, neural networks, and logistic regression. What I am trying to understand ...
user avatar
  • 6,019
5 votes
2 answers
2k views

Can we correctly identify all the non-zero coefficients in the linear regression model?

I have a conceptual question regarding linear regression. Assume our model is correct, i.e., the response variable $Y$ is indeed coming from the model $$Y=\beta_0+\beta X+\epsilon.$$ Here $X$ is ...
user avatar
  • 6,407
0 votes
1 answer
168 views

Hosmer–Lemeshow test - Best Model

I have 3 different models and I do the Hosmer–Lemeshow test. I have a p-value and a Chi2 value. How can I know which model fits the best my data? Khi-2 || Pr > Khi-2 12.04 || 0.19 7.47 || 0.71 4....
user avatar
4 votes
0 answers
2k views

How to find marginal effect of restricted cubic spline

I'm trying to figure out how to find the marginal effect of an interaction term from a restricted cubic spline in a non-linear model. The post Nonlinear effect in an interaction term is a good start ...
user avatar
  • 241
11 votes
3 answers
20k views

Ranking features in logistic regression

I used Logistic Regression. I have six features, I want to know the important features in this classifier that influence the result more than other features. I used Information Gain but it seems that ...
user avatar
  • 211
5 votes
1 answer
1k views

How to apply a model on dataset with missing data?

This question is similar to Missing input value during prediction of a generalized linear model. Consider the following scenario: I fitted a linear regression model on a training dataset with ...
user avatar
0 votes
0 answers
56 views

How to choose between data-driven pattern or intuition?

I am performing a multivariate logistic regression (This could very well be any other kind of regression method) to study the effect of some predictor variables on the probability of event. I have ...
user avatar
4 votes
2 answers
3k views

What to do with non-normality and heterogeneous variances in two-way ANOVA when transformations do not work?

I'm conducting a Two-Way ANOVA with my two factors being Sex and Cohort. I have data from two cohorts of subjects, with each cohort consisting of males and females that were measured on one response ...
user avatar
  • 41
2 votes
1 answer
300 views

Is my understanding of regularized logistic regression correct?

I learned that regularized logistic regression helps prevent the model from over-fitting the data. I understand that the function is still technically a high-order polynomial, but the effect is ...
user avatar
1 vote
2 answers
408 views

Using a dichotomous categorical variable that has an underlying continuous dichotomy in a multiple regression?

FINAL EDIT I just found a good answer to this question in another thread in the forum here, therefore, I think this question could be closed. Thanks so much for your help and the clarification about ...
user avatar
  • 11
1 vote
1 answer
190 views

logistic regression predictive modeling

I would like to use a logistic regression for estimating the parameters of the logit function by using the maximum likelihood estimate. This amounts to minimizing the log-loss function, instead of ...
user avatar
1 vote
1 answer
2k views

Test/Measure for Rank Ordering a Logistic Regression model, invariant to event rate and population size

I have a model whose purpose is to rank order event risk - the output of which is split into twentiles (which have been based off the benchmark data). Currently, I'm using Somers' D calculated on ...
user avatar
4 votes
3 answers
578 views

How to best model interaction effect of two continuous predictor variables?

Consider the following problem: In a logistic regression model, we believe that two continuous predictor variables $X_1$ and $X_2$ impact the probability of event. It is hypothesized that the ...
user avatar
1 vote
1 answer
138 views

Choosing which of the independent variables to enter into multinomial regression

I am performing regression analysis only very newly, so am practically lacking skills! It would be greatly appreciated getting your guidance pls. I have 50 observations of 26 IV (categorical ) and ...
user avatar
  • 13
0 votes
1 answer
1k views

Running regressions where coefficients change over time

I'm trying to predict monthly stock price returns using 93 features that I think may be relevant. I have data for these features from 1990 to 2015. For each month from 1990 to 2015 I run a ...
user avatar