Questions tagged [regression-strategies]
Regression Modeling Strategies
289
questions
11
votes
2
answers
1k
views
How to start building a regression model when the most strongly associated predictor is binary
I have data set containing 365 observation of three variables namely pm, temp and rain. Now ...
1
vote
0
answers
50
views
Use R to perform a logistic regression and cross-validate in a small sample?
I have a small sample (n=69, 35 with diseases, 34 without disease). I have around 20 variables that came up as significant using indiscriminate univariate regression. Of these 20 variables, several ...
- 11
2
votes
1
answer
159
views
Estimation of treatment effect when there is an unknown and variable coverage of the population
I am not sure if I am using the correct terminology, something must be written about the following problem, but I cannot find it by searching.
I am presently analyzing data about the effect of ...
- 64.7k
1
vote
0
answers
997
views
Why does gvlma() give contradictory results for linear model assumptions?
I have fitted a linear model and I am checking the assumptions for which I get this diagnostic plot:
From this diagnostic plot there seems to be an increasing variance in the residuals for increasing ...
- 93
2
votes
0
answers
369
views
Do the problems of stepwise variable selection exist in FA, PCA, SEM?
Note: This is a revision of my original question.
I have read the critique of stepwise variable selection and "all possible subsets regression" by Professor Frank Harrell here.
Are factor analysis, ...
- 96
1
vote
1
answer
84
views
Discovering transformations and interactions
I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the ...
- 13
0
votes
0
answers
32
views
Discovering interactions and transformations [duplicate]
I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the ...
- 13
2
votes
0
answers
43
views
Is having more features definitely equal to having a higher chance of overfitting?
I am doing a EEG data classification problem. Currently I am using the ANOVA test to help me select K best input features (with K a parameter to tune) and feeding the selected features into a logistic ...
- 203
2
votes
1
answer
2k
views
Alternative to linear regression
I need to run hundreds of linear regression models, with the same set of independent variables, but with varying dependent variables. I have checked normality for a few dozens. Some are normally ...
- 685
8
votes
1
answer
7k
views
Goodness of fit, predictive power, discrimination
I'm making a couple of logistic regression based predictive models and intend to compare them and see which is "best". "best" here is obviously ill-defined, but as I'm looking for common metrics for ...
- 843
2
votes
1
answer
419
views
Problems with zero values when testing for linearity of the logit
From Field's 'Discovering Statistics using SPSS' I read that you should test for linearity of the logit when using a logistic regression. The process explained in the book covers the natural log ...
- 163
1
vote
3
answers
2k
views
Logistic Regression Cutoff Values for Multiple Models
I understand that once the logistic regression model has output probabilities, a cutoff value for classifying probabilities of new observations is decided for a model to optimize some metric like ...
- 71
11
votes
2
answers
8k
views
Using LASSO only for feature selection
In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization.
My question: do people normally use ...
- 111
0
votes
1
answer
174
views
Does the p value for logistic regression depends on odds ratio or logit?
The p value in a logistic regression is given for the B estimates. But what if I'm reporting odds ratio instead of logit. Do the same p values apply?
- 61
3
votes
0
answers
1k
views
R: Quadratic Regression with interaction: when to center?
I have a statistical question.
I have data from an experiment with two conditions (dichotomous IV: 'condition'). I also want to make use of another IV which is metric ('hh'). My DV is also metric ('...
- 41
0
votes
1
answer
162
views
Optimal Cutpoint for Predicted Results from Kaplan Meier and Cox Regression
Is there anyway to get the optimal cutpoint for predicted survival probabilities of the aforementioned survival analysis approaches?
Something like the optimal cutpoint from ...
- 15
2
votes
1
answer
93
views
Is it normal for logistic regression, to have predictors which have good Wald's Chi Sq, but still bad performance?
I have trying to build a logistic model with some 10 variables. All of those variables have Wald Chi sq value<500. All are highly significant p-value<0.0001.
Event rate is about 3%.
The ...
- 717
2
votes
2
answers
1k
views
predict from a Cox model with beta coefficients
I fitted a Cox PH model in R with the survival package and the coxph function.
I get the beta estimates from this model.
How can I use these coefficients to ...
- 1,333
0
votes
1
answer
330
views
Negative coefficients for ordinal logistic regression in R
I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in "rms"...
- 21
17
votes
4
answers
8k
views
Why does propensity score matching work for causal inference?
Propensity score matching is used for make causal inferences in observational studies (see the Rosenbaum / Rubin paper). What's the simple intuition behind why it works?
In other words, why if we ...
- 1,254
1
vote
0
answers
1k
views
How to use aregImpute "group" argument?
Can someone provide an example of using the "group" argument with aregImpute()? I see that group=NULL is the default, but my data include a few factor variables with levels with <5 observations. My ...
- 11
9
votes
1
answer
5k
views
How to make calibration plot for survival data without binning data?
To make a calibration plot for survival probabilities estimated from a Cox model, one can divide the estimated risk into groups, calculate the average risk within a group, and then compare this to the ...
- 2,262
0
votes
1
answer
3k
views
Logistic Regression on Time-dependent Predictors
I would like to know if I can apply the techniques, like say Logistic Regression, to data whose variables/predictors are 'indexed' by time. Or if not, what techniques are appropriate to use in these ...
- 1
0
votes
1
answer
1k
views
Why can't we use linear regression for discrete Y?
This text is from An Introduction to Statistical Learning with Applications in R(by • Gareth James • Daniela Witten • Trevor Hastie • Robert Tibshirani)
List item
Can anyone help me by making linear ...
- 239
1
vote
1
answer
580
views
linearity test for cublic splines
Consider two models in which a continuous variable is modeled as a restricted cubic spline (RCS) or entered linearly. If one carries out a test for linearity, why are the degrees of freedom for the ...
- 2,262
0
votes
1
answer
48
views
What should be the ratio between number of cases and attributes in multivariate regression?
Is there any way to determine if it is feasible to perform a multivariate regression based on a given number of samples and attributes?
For example I have a data set with 6 cases , 30 attributes and ...
- 11
7
votes
1
answer
5k
views
To select variables or not in logistic regression
I am trying to find predictors for an outcome. I was taught to perform univariate analyses & put significant variables into a multivariate logistic regression model. Then I remove variables one by ...
- 111
1
vote
1
answer
92
views
Choosing number of samples to train a model
(On behalf of a colleague)
I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
- 113
3
votes
1
answer
309
views
To use a two-sample t-test or to use a Mann-Whitney U-test on spatial data?
I have obtained some mean summer temperature records (June 1st- September 30th) for all years within the 1930s (1930-1939) and I have obtained summer temperature records for all years within the ...
- 83
3
votes
1
answer
1k
views
Randomizing Class Labels during classification to asses the feature selection results
I have a binary classification problem with thousands of variables and less than a hundred data points and class labels. The class is imbalanced (24 positive 51 negative samples). I have selected some ...
- 744
1
vote
2
answers
1k
views
Which balance strategy to learn from my very imbalanced dataset?
I'm using a deep learning approach on a dataset made of ~20 millions of elements, where each element has a TRUE or FALSE label.
This dataset unfortunately is veeery imbalanced: I've 98% of falses and ...
- 692
7
votes
2
answers
173
views
What's wrong with data-guided modeling in regression?
In the Regression Modelling Strategies of Frank Harrell, section 4.1, if I understood correctly, it is not recommendred to using the data to decide how to represent a predictor in a regression model (...
- 2,376
2
votes
2
answers
6k
views
Minimum number of observations needed for penalized regression?
I'm wondering what should be the minimum sample size to perform ridge, lasso or elastic net regression. I have a binomial outcome that I want to relate with a set of features (18 features in some ...
12
votes
3
answers
9k
views
In general, does normalization mean to normalize the samples or features?
I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
- 121
8
votes
2
answers
439
views
Logistic Regression with (Normal) Distributions for Independent Variables
Consider the logistic regression where $Y_i \in {0,1}$ are dependent variable observations and $X_i \in \mathbb{R}$ are the independent variables.
However we do not observe the $X_i$ themselves. ...
- 622
13
votes
2
answers
7k
views
When is logistic regression suitable?
I'm currently teaching myself how to do classification, and specifically I'm looking at three methods: support vector machines, neural networks, and logistic regression. What I am trying to understand ...
- 6,019
5
votes
2
answers
2k
views
Can we correctly identify all the non-zero coefficients in the linear regression model?
I have a conceptual question regarding linear regression.
Assume our model is correct, i.e., the response variable $Y$ is indeed coming from the model
$$Y=\beta_0+\beta X+\epsilon.$$
Here $X$ is ...
- 6,407
0
votes
1
answer
168
views
Hosmer–Lemeshow test - Best Model
I have 3 different models and I do the Hosmer–Lemeshow test. I have a p-value and a Chi2 value. How can I know which model fits the best my data?
Khi-2 || Pr > Khi-2
12.04 || 0.19
7.47 || 0.71
4....
4
votes
0
answers
2k
views
How to find marginal effect of restricted cubic spline
I'm trying to figure out how to find the marginal effect of an interaction term from a restricted cubic spline in a non-linear model. The post Nonlinear effect in an interaction term is a good start ...
- 241
11
votes
3
answers
20k
views
Ranking features in logistic regression
I used Logistic Regression. I have six features, I want to know the important features in this classifier that influence the result more than other features. I used Information Gain but it seems that ...
- 211
5
votes
1
answer
1k
views
How to apply a model on dataset with missing data?
This question is similar to Missing input value during prediction of a generalized linear model.
Consider the following scenario:
I fitted a linear regression model on a training dataset with
...
- 581
0
votes
0
answers
56
views
How to choose between data-driven pattern or intuition?
I am performing a multivariate logistic regression (This could very well be any other kind of regression method) to study the effect of some predictor variables on the probability of event. I have ...
- 581
4
votes
2
answers
3k
views
What to do with non-normality and heterogeneous variances in two-way ANOVA when transformations do not work?
I'm conducting a Two-Way ANOVA with my two factors being Sex and Cohort. I have data from two cohorts of subjects, with each cohort consisting of males and females that were measured on one response ...
- 41
2
votes
1
answer
300
views
Is my understanding of regularized logistic regression correct?
I learned that regularized logistic regression helps prevent the model from over-fitting the data. I understand that the function is still technically a high-order polynomial, but the effect is ...
- 23
1
vote
2
answers
408
views
Using a dichotomous categorical variable that has an underlying continuous dichotomy in a multiple regression?
FINAL EDIT
I just found a good answer to this question in another thread in the forum here, therefore, I think this question could be closed. Thanks so much for your help and the clarification about ...
- 11
1
vote
1
answer
190
views
logistic regression predictive modeling
I would like to use a logistic regression for estimating the parameters of the logit function by using the maximum likelihood estimate. This amounts to minimizing the log-loss function, instead of ...
- 25
1
vote
1
answer
2k
views
Test/Measure for Rank Ordering a Logistic Regression model, invariant to event rate and population size
I have a model whose purpose is to rank order event risk - the output of which is split into twentiles (which have been based off the benchmark data).
Currently, I'm using Somers' D calculated on ...
- 19
4
votes
3
answers
578
views
How to best model interaction effect of two continuous predictor variables?
Consider the following problem:
In a logistic regression model, we believe that two continuous
predictor variables $X_1$ and $X_2$ impact the probability of event.
It is hypothesized that the ...
- 581
1
vote
1
answer
138
views
Choosing which of the independent variables to enter into multinomial regression
I am performing regression analysis only very newly, so am practically lacking skills! It would be greatly appreciated getting your guidance pls.
I have 50 observations of 26 IV (categorical ) and ...
- 13
0
votes
1
answer
1k
views
Running regressions where coefficients change over time
I'm trying to predict monthly stock price returns using 93 features that I think may be relevant. I have data for these features from 1990 to 2015. For each month from 1990 to 2015 I run a ...
