Questions tagged [regression-strategies]

Regression Modeling Strategies

287 questions
Filter by
Sorted by
Tagged with
16 views

Repeated measures ordinal outcomes

I have an outcome that is measured at two time points, baseline and 1 year after. The outcome is on an ordinal scale: 3 = better, 2 = somewhat better, 1 = somewhat worse, and 0 = worse. I'm interested ...
8 views

Converting a code for 5-fold cross validation to stratified 5-fold cross validation for continuous target variable

Do you know how I can convert my code so it can do stratified 5-fold cross validation on a continuous target? df['score'] or y is a continuous variable. ...
375 views

Logistic Regression with (Normal) Distributions for Independent Variables

Consider the logistic regression where $Y_i \in {0,1}$ are dependent variable observations and $X_i \in \mathbb{R}$ are the independent variables. However we do not observe the $X_i$ themselves. ...
20 views

(Stratified) 5-fold cross validation for 2D tensor and real-valued target regression using sklean train_test_split method

In classification problem, when we want to do stratified 5-fold cross validation, we pick the target and use train_test_split using something like below: ...
402 views

Compare the results of two canonical correlation analyses (CCA)

I have four datasets: morphological measurements for a set of species (M1), ecological measurements for the same set of species (E1), morphological measurements for a second set of species (M2), and ...
240 views

Does full subset selection suffer from the same handicaps as stepwise regression?

Let's assume $p$ potential predictor variables $X_1,...,X_p$ and a single dependent variable $Y$. Now I evaluate the performance of all possible linear models considering all possible combinations of ...
134 views

how can I obtain a beta value for three way interaction term in a logistic regression

I am using the RMS package in R to conduct a logistic regression that contains a three-way interaction. As part of my modelling approach, I have conducted chunk tests of the interaction (using Wald ...
528 views

External validation of a published Cox PH model

My aim is to externally validate a risk prediction model published in the medical literature that is based on a Cox regression model. I have a dataset with all the variables from the score. I read ...
206 views

Univariate and multivariate regression

I found a research article Nurses’ reports of staffing adequacy and surgical site infections: A cross-sectional multi-centre study and I want to know the reason why they used regression for their ...
94 views

Logit Regression and F-test: Can I apply the f statistic when variables are non-normal and the output is binary?

I want to do a univariate analysis on a set of variables to see which predict a binary outcome. I want to discard some of them before performing logistic regression. I am trying to understand if I can ...
3k views

Relative importance of variables in Cox regression

I've understood that relative importance of predictors is a tricky question. Suggested methods range from very complex models to very simple variable transformations. I've understood that the ...
69 views

Meaning of interaction with %ia% in rms? Three-way interaction?

In this very illustrative post on evaluating added value of predictors by Frank Harrell, he codes a logistic regression model as such: ...
98 views

Estimating regression optimism using the bootstrap

I am estimating optimism bias in for example risk predictions. A method for doing that is described by Frank Harrell and implemented in the R package rms. I am ...
24k views

Can a random forest be used for feature selection in multiple linear regression?

Since RF can handle non-linearity but can't provide coefficients, would it be wise to use random forest to gather the most important features and then plug those features into a multiple linear ...
44 views

I am a non-statistician. I have to write a non-English article about my data analysis workflow for a particular epidemiological regression analysis that I conducted. The article will cover my workflow ...
7 views

How to interpret the effect of age on predicted wage for different Majors

I am conducting a multiple regression analyses to find the premiums on predicted log of wage associated with majoring in STEM vs non STEM degree. I take lnwage as the dependent variable and run ...
39 views

Effect size derived from LME longitudinal model: the statistical findings projected back down onto a group of people

I have been studying the change in a metric X with a linear mixed effect model. I have built this model in a multivariate setting, so I can see how each of my covariates (Time, sex, age) affect X. ...
14k views

Logistic Regression with regression splines in R

I have been developing a logistic regression model based on retrospective data from a national trauma database of head injury in the UK. The key outcome is 30 day mortality (denoted as "Survive&...
88 views

How to assess variable to transform in multiple regression?

I have a multiple regression model and when I check its residuals vs fitted I have determined a transformation of some kind needs to take place ... but I don't know which variable to start with (4 ...
18 views

Modelling strategies for analyzing an effect of a predictor through higher hierarchical level

What strategies can be considered when a predictor's direct effect can not be measured directly due to unmeasured confounding? However, data has a hierarchical structure (patients within regions) that ...
43k views

What is the benefit of breaking up a continuous predictor variable?

I'm wondering what the value is in taking a continuous predictor variable and breaking it up (e.g., into quintiles), before using it in a model. It seems to me that by binning the variable we lose ...
7 views

Correlating variables taken from different samples

Considering a hypothetical example, we have samples from Twitter and Facebook across US counties during the same time period. Say we asked a different question on the different platforms. People on ...
42 views

Applying count models with rate responses

How do you apply count models to data which is count in nature, but a rate in reality? In such cases, r can handle this to a certain extent, depending on the model, but what is the correct way to ...
2k views

Reporting the effect of a predictor in a logistic regression fitted with a restricted cubic spline

I have been playing around with using restricted cubic splines using the RMS package. Output below. ...
288 views

Boruta Algorithm for Logistic Regression?

Is it okay to use a Boruta algorithm to select features for a logistic regression? I read several sources, including the source package as well as this site explaining what Boruta does. My ...
3k views

How to compare (probability) predictive ability of models developed from logistic regression?

I know some well-known measures are $c$ statistic, Kolmogorov-Smirnov $D$ statistic. However, as far as I know, those statistics take into account only of the rank order of the observations, and is ...
193 views

Why do we even bother running regression models?

I'm working through regression with Intro to Statistical Learning by Hastie, Witten, James and Tibshirani. They break down regression into stages: data cleaning and processing, model building and ...
16 views

The procedure of adding interaction terms in regression models

What is the more sensible way to add interaction terms in regression models? I have a basic model which includes only the main effects. To add interactions to the basic model, do I add all of them at ...
44k views

How should I check the assumption of linearity to the logit for the continuous independent variables in logistic regression analysis?

I am confused with the assumption of linearity to the logit for continuous predictor variables in logistic regression analysis. Do we need to check for the linear relationship while screening for ...
343 views

Brier score of calibrated probs is worse than non calibrated probs

The question is related to probability calibration and Brier score I have faced with the following issue. I have Random forest binary classifier and then I apply isotonic regression to calibration of ...
3k views

Handling missing data in logistic regression

I'm trying to do logistic regression, but I can't seem to get the results I want. I have 6 columns of data (one dependent and 5 independent binary variables) and about 100 rows. The problem with my ...
247 views

When LASSO selects only parts of a categorical variable?

I want to use LASSO to construct a model and then run a logistic regression on the variables LASSO selects. However, LASSO selects only parts of some categorical variables that I put into it. Does ...
102 views

Using "moderately" correlated variables to select controls for a LASSO regression?

In medicine we often have a disease status as an outcome variable and a lot of independent variables in which we want to see if there is some connection. Traditionally, baseline characteristics such ...
79 views

Setting contrasts for 10-level categorical variable

I have survey data on income and support for environmental protection. Income is a continuous variable that I have broken up into deciles. I have a hypothesis that support for protection ('Agree') ...
401 views

probability calibration and Brier score

Assume that I have a binary classification problem. The outcome from classification I am mostly interested in is the well-calibrated probabilities. The first way to check this is the calibration plot (...
4k views

GLM with continuous data piled up at zero

I am trying to run a model to estimate how well catastrophic illnesses such as TB, AIDS etc affect spending on hospitalization. I have "per hospitalization cost" as the dependent variable and various ...
5k views

Regression with categorical predictors - use only some dummy variables [duplicate]

I am working on a regression and I have a factor variable "Marital Status" Marital status has 5 levels: Single, Married, Divored, Widowed, Other (don't ask me what constitutes someone being an 'other'...
43k views

Evaluating logistic regression and interpretation of Hosmer-Lemeshow Goodness of Fit

As we all know, there are 2 methods to evaluate the logistic regression model and they are testing very different things Predictive power: Get a statistic that measures how well you can predict the ...
89 views

How to validate Generalized Least Square model for longitudinal response

I have a dataset with body weights before and in the follow-up visits after surgery, for a group of patients with obesity. Our goal is to fit a model to predict weight loss throughout the follow-up. ...
4k views

Should final (production ready) model be trained on complete data or just on training set?

Suppose I trained several models on training set, choose best one using cross validation set and measured performance on test set. So now I have one final best model. Should I retrain it on my all ...
32 views

We have developed a basic Regression framework where we try to build models for over 100 configs(stored in a file). To run : ...
45k views

Overfitting a logistic regression model

Is it possible to overfit a logistic regression model? I saw a video saying that if my area under the ROC curve is higher than 95%, then its very likely to be over fitted, but is it possible to ...
95 views

OLR with rms: proportional odds assumption

I am fitting an ordinal logistic regression model with rms package. my data involves a three-level ordered outcome (see ...
2k views

How to visualize a significant interaction between two linear predictors using the rms package?

Two linear predictors interact significantly (see below). How can I visualize this interaction in a plot? ...
179 views

Excluding the effect of control variables in the assessment of a logistic regression model

I have a logistic regression model with ten independent variables of which two are included as controls. While their inclusion is necessary for correctly assessing the coefficients of the other ...
854 views

Does LASSO suffer from the same problems stepwise regression does?

Stepwise algorithmic variable-selection methods tend to select for models which bias more or less every estimate in regression models ($\beta$s and their SEs, p-values, F statistics, etc.), and are ...
903 views

Is it reasonable to drop an interaction term?

I'm regressing a model $Y = X_1 + X_2 + X_1X_2$ and the result turns out that none of them are significant. However, if I drop the interaction term, $X_1$ becomes significant. Is it ok to drop the ...
33 views

Combining mean differences in regressors and significant prediction and moderation in multiple regression

I am analyzing a survey about career choices with an eye toward demonstrating sex differences in: 1) the means for factors that might be related to career choice (e.g., differences by sex in ...