Questions tagged [regression-strategies]

Regression Modeling Strategies

Filter by
Sorted by
Tagged with
8 votes
6 answers
3k views

Dealing with non-normal distribution in "big" datasets, when do we throw out the CLT?

Apologies from the go as this question comes from an absolute newbie and will definitely not satisfy a lot of the detail required. Hence, your guidance in providing you the right information to allow ...
user avatar
  • 438
12 votes
3 answers
4k views

GLM with continuous data piled up at zero

I am trying to run a model to estimate how well catastrophic illnesses such as TB, AIDS etc affect spending on hospitalization. I have "per hospitalization cost" as the dependent variable and various ...
user avatar
  • 121
1 vote
1 answer
367 views

Confidence intervals for predictors in multivariate logistic regression

I've got a question. I am dealing with medical data which contain 5 predictors and 1 binary outcome. When I try to classify the data using all 5 predictors I get 0.84 area under roc-curve which is ...
user avatar
  • 611
8 votes
3 answers
5k views

Internal validation via bootstrap: What ROC curve to present?

I am using the bootstrap approach for internal validation of a multivariate model built with either standard logistic regression OR elastic net. The procedure I use is as follows: 1) build model ...
user avatar
  • 81
5 votes
3 answers
1k views

Removing attributes with few observations in R

I have roughly 15 variables / attributes characterizing 6k customers in my data set. As they are categorical I have transformed them into 1 attribute for each possible value (1-out-of-K coding). An ...
user avatar
  • 4,726
0 votes
0 answers
435 views

Correct methodology to repeat testing of classifier to get good estimate of performance

I'm having trouble with a basic machine learning methodology question. I understand the concept of not using the same data to both train and evaluate a classifier, and furthermore when there are ...
user avatar
9 votes
5 answers
12k views

Logistic Regression on Big Data

I have a data set of around 5000 features. For that data I first used Chi Square test for feature selection; after that, I got around 1500 variables which showed significance relationship with the ...
user avatar
  • 305
1 vote
1 answer
436 views

Interpretation with training and test set with standardized variables

I've standardized all the variables (even the response variable) and then I've split my data into a training and test part. And for example, I've got the following model based on my TRAINING set: y = ...
user avatar
  • 295
1 vote
1 answer
196 views

Can anyone suggest me articles where they have used multivariate logistic regression models and explored in detail about the role of each predicor?

I am doing logistic regression analysis using multiple predictors for a binary outcome.I had about 10 predictors and tried to find the best model using 'glmulti' package in R. I have got a significant ...
user avatar
  • 519
0 votes
0 answers
260 views

validation of a Zero Adjusted Gamma model

I am using a Zero Adjusted Gamma regression (two-part ZAGA) model to estimate the effect of a psychometric categorical factor ("expected recovery", with 3 levels) on cost associated with treatment ...
user avatar
  • 151
2 votes
1 answer
4k views

Test overfitting of logistic regression with limited volume

I have a set of samples with two labels red and black. I can build a logistic regression model to predict the label colour. Once a model is built, I would like to test whether it is overfitting or not....
user avatar
  • 133
5 votes
1 answer
3k views

How to choose the right number of parameters in Logistic Regression?

I am studying Andrew Ng's Machine Learning lecture notes. I understand either we can manually choose the number of parameters, or we can use regularization to make it correctly fit. I was wondering ...
user avatar
  • 815
2 votes
1 answer
5k views

How to remove non-significant interaction(s) from meta-regression (rma) model

According to the rule of marginality, I should remove all non-significant interactions from model to achive simplicity (MInimal Adequate Model). I am familiar with removing such interactions from ...
user avatar
1 vote
0 answers
31 views

100mn rows of "events" with 30 or so attributes, want to understand what affects one particular attribute - how? which software?

I have about 50-100 million rows of data of interesting "events". Each of these rows has about 32 attributes. One of these attributes is how much money we made :-) What's the best way to make sense ...
user avatar
3 votes
3 answers
3k views

What problems do non-normality in predictor variables cause for a multiple regression analysis?

I am talking about a situation in which I have several continuous predictor variables predicting a continuous outcome. One of the predictors has a very non-normal distribution and has some wild ...
user avatar
10 votes
5 answers
13k views

Treating ordinal variables as continuous for regression problems

In the social sciences I have encountered that it is common to treat ordinal variables as continuous, for example variables originating from rating or Likert scales (strongly disagree, disagree, agree,...
user avatar
  • 569
4 votes
3 answers
1k views

Selecting most important variable based on individual p-value vs. partial $R^2$

I'm trying to solve a problem where the goal is to find an association between children's cortisol values (y) against their mother's weekly cortisol averages (...
user avatar
  • 41
3 votes
1 answer
2k views

What methods exist for finding optimal splits to discretize continuous data with respect to a target variable

I'm doing some research into methods for discretizing a continuous variable coupled with a binary target variable to find the optimal split points to maxamise a measure of impurity (gini/entropy). ...
user avatar
  • 513
2 votes
1 answer
429 views

Compare the results of two canonical correlation analyses (CCA)

I have four datasets: morphological measurements for a set of species (M1), ecological measurements for the same set of species (E1), morphological measurements for a second set of species (M2), and ...
user avatar
3 votes
0 answers
585 views

Permuting the formula argument to Hmisc:aregImpute

In Frank Harrell's RMS Short Course today, I became aware that multiple imputation with Hmisc:aregImpute is not invariant to the ordering of terms in its formula ...
user avatar
9 votes
2 answers
2k views

Selecting the number of sparse principal components to include in regression

Does anyone have experience with approaches for selecting the number of sparse principal components to include in a regression model?
user avatar
0 votes
0 answers
984 views

Should I include demographics in my regression?

I am doing a study to try to explain certain behaviour using 6 predictors. I have used a questionnaire to gather the data. The questionnaire included a demographics section eg age, gender, job ...
user avatar
17 votes
4 answers
21k views

Which variables explain which PCA components, and vice versa?

Using this data: head(USArrests) nrow(USArrests) I can do a PCA as thus: plot(USArrests) otherPCA <- princomp(USArrests) ...
user avatar
3 votes
4 answers
940 views

Can the use of dummy variables reduce measurement error?

If the continuous variables are measured with error, can the use of dummy variables mitigate the problem? For instance, IQ measures intelligence with error. So will using a dummy of high, medium, low ...
user avatar
  • 55
5 votes
2 answers
6k views

LASSO to identify important variables in ordered logistic regression?

I have spent two days grappling with this question, and the range of ambiguous answers online has driven me to ask. I am working with R. I have a dataset where my dependent variable is an ordered ...
user avatar
  • 51
2 votes
0 answers
265 views

Ridge regression not appropriate for collinearity caused by mathematical constraints on the data

In this paper: Use of the Bootstrap and Cross-Validation in Ridge Regression Author(s): Nancy Jo Delaney and Sangit Chatterjee Source: Journal of Business & Economic Statistics, Vol. 4, No. 2 (...
user avatar
  • 1,583
5 votes
1 answer
251 views

Does full subset selection suffer from the same handicaps as stepwise regression?

Let's assume $p$ potential predictor variables $X_1,...,X_p$ and a single dependent variable $Y$. Now I evaluate the performance of all possible linear models considering all possible combinations of ...
user avatar
  • 1,424
3 votes
1 answer
1k views

Box Cox Transformation with swift

I am trying to do a box-cox transformation with swift. I have a dependent variable, annual foreign sales of companies (in US\$ thousands) which contains zeros, for a set of panel data. I have been ...
user avatar
36 votes
5 answers
47k views

Overfitting a logistic regression model

Is it possible to overfit a logistic regression model? I saw a video saying that if my area under the ROC curve is higher than 95%, then its very likely to be over fitted, but is it possible to ...
user avatar
100 votes
8 answers
45k views

What is the benefit of breaking up a continuous predictor variable?

I'm wondering what the value is in taking a continuous predictor variable and breaking it up (e.g., into quintiles), before using it in a model. It seems to me that by binning the variable we lose ...
user avatar
  • 1,531
2 votes
0 answers
290 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
user avatar
4 votes
2 answers
4k views

Which model should I use to fit my data ? ordinal and non-ordinal, not normal and not homoscedastic

Here is the kind of data I have: I have two predictor variables: discrete non-ordinal --> c('a', 'b', 'c') discrete ordinal --> ...
user avatar
  • 329
2 votes
1 answer
4k views

fastbw with rule="p" in R's rms package: why do results depend on number of covariates?

I've been trying to use the fastbw function from the rms package in R to perform logistic regression with backward selection, with p-values as exclusion criterion (I am well aware of the arguments ...
user avatar
25 votes
5 answers
12k views

When is quantile regression worse than OLS?

Apart from some unique circumstances where we absolutely must understand the conditional mean relationship, what are the situations where a researcher should pick OLS over Quantile Regression? I don'...
user avatar
4 votes
3 answers
285 views

Applying an interaction term to all the IVs

I have a linear model with 6 IVs and would like to analyze the effect of an interaction term applied to all the IVs. To illustrate, let's say we're predicting the Win/Loose ratio of NBA basketball ...
user avatar
19 votes
5 answers
14k views

Can I ignore coefficients for non-significant levels of factors in a linear model?

After seeking clarification about linear model coefficients over here I have a follow up question concerning non-signficant (high p value) for coefficients of factor levels. Example: If my linear ...
user avatar
27 votes
1 answer
2k views

Appropriate residual degrees of freedom after dropping terms from a model

I am reflecting on the discussion around this question and particularly Frank Harrell's comment that the estimate for variance in a reduced model (ie one from which a number of explanatory variables ...
user avatar
  • 16.6k
13 votes
3 answers
15k views

Model Selection: Logistic Regression

Suppose we have $n$ covariates $x_1, \dots, x_n$ and a binary outcome variable $y$. Some of these covariates are categorical with multiple levels. Others are continuous. How would you choose the "best"...
user avatar
  • 141
14 votes
3 answers
14k views

Testing nonlinearity in logistic regression (or other forms of regression)

One of the assumption of logistic regression is the linearity in the logit. So once I got my model up and running I test for nonlinearity using Box-Tidwell test. One of my continuous predictors (X) ...
user avatar
  • 321

1 2 3 4 5
6