Questions tagged [regression-strategies]

Regression Modeling Strategies

Filter by
Sorted by
Tagged with
1
vote
1answer
1k views

Testing if variables have a non-linear relationships with the dependent variable [duplicate]

How can I find evidence that a independent variable has a non-linear relationship with the dependent variable? Can I possibly achieve this by squaring all the independent variables and estimate a ...
3
votes
1answer
619 views

Calculating goodness of fit and choosing the right model (R)

I'm trying to evaluate the value of an object, depending on his characteristics. In order to do this, I'm building the following regression model price ~ ., using ...
2
votes
1answer
895 views

Is it reasonable to drop an interaction term?

I'm regressing a model $Y = X_1 + X_2 + X_1X_2$ and the result turns out that none of them are significant. However, if I drop the interaction term, $X_1$ becomes significant. Is it ok to drop the ...
10
votes
2answers
11k views

Equation of a fitted smooth spline and its analytical derivative [duplicate]

I need to fit a spline function to a data set. I tried with bs, ns and smooth.spline. In my ...
1
vote
1answer
691 views

Creating an interaction term with 2 continuous variables: What to do?

I want to create an interaction term in SPSS on two continuous variables (ticket price and household income) in order to use this interaction term in a multiple regression model and test whether my ...
1
vote
2answers
5k views

Regression with categorical predictors - use only some dummy variables [duplicate]

I am working on a regression and I have a factor variable "Marital Status" Marital status has 5 levels: Single, Married, Divored, Widowed, Other (don't ask me what constitutes someone being an 'other'...
5
votes
1answer
516 views

When should I use feature selection and when should I use dimensionality reduction techniques?

When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use ...
1
vote
2answers
794 views

test of correlation among binary variables prior to running logistic regression analysis

I am running a logistic regression analysis with binary variables on SPSS: dependent variable: preterm birth (Y/N) independent variables: hypertension (Y/N), diabetes (Y/N), C section (Y/N), ...
7
votes
4answers
6k views

Nonlinear effect in an interaction term

If you have B, which is a 0/1 outcome variable, S, which is a continuous variable, and ...
3
votes
1answer
828 views

The best model of an AICc-based model selection on a very small sample has an high number of predictors: does it make any sense?

I'm working with a very small sample size (N=14) and I'm using AICc to identify the most parsimonious model using a large number of possible predictors. Unexpectedly the best model has six predictors! ...
1
vote
0answers
959 views

Estimation of theta for IHS (inverse hyperbolic sine) transformation

I am trying to use the IHS transformation to correct for heteroskedasticity in a Tobit model. The main references have been Pence (2006) and Burbidge et al. (1988). I have noticed that the convention ...
2
votes
2answers
806 views

Multiple test correction for categorical variables with many levels

I am creating a binary logistic regression model with the inputs as a single categorical variable with 100 levels. My goal is to find which level of the categorical variable is most likely to result ...
12
votes
3answers
22k views

Should I remove non-significant variables from my regression model

I have run a multiple linear regression using stepwise regression to select the best model, however the best model returned has a non-significant variable. When I remove this the AIC value goes up ...
0
votes
1answer
66 views

Cross validation on clinical datasets [closed]

I am very new to R programming. In my project I need to perform a Cross validation for the clinical datasets (small). I want to know what will be the results. I am unable to recognize the results. I ...
6
votes
4answers
3k views

Linear regression - is a model "useless" if $R^2$ is very small?

Given a complex output which depends on many underlying factors, I am given 3 explanatory variables and about 10K data points and the task to assess their impact on the output. The OLS model is very ...
14
votes
1answer
4k views

What is the difference between 'hypothesis testing' and 'model selection'?

In literature, both terms are often used synonymously or interwoven. I am now trying to find a clear distinction between both terms. From my point of view, a hypothesis is usually expressed via a ...
10
votes
3answers
2k views

What are criteria and decision making for non-linearity in statistical models?

I hope that the following general question makes sense. Please keep in mind that for the purposes of this particular question I'm not interested in theoretical (subject domain) reasons for introducing ...
1
vote
2answers
10k views

Logistic regression - how good is my model? [duplicate]

I am a beginner in ML so apologize in advance if this sounds silly. I did a logistic regression on a real data set and I am having problems measuring how well my model fits. I still don't understand ...
2
votes
1answer
482 views

Fractional polynomials vs GAMs

I have been analyzing panel data for a while now using different methods (Generalized Linear Models, fractional polynomials and GAMs). If we just ignore GMMs for now, I have come to find that ...
10
votes
2answers
953 views

Best method to create growth charts

I have to create charts (similar to growth charts) for children of ages 5 to 15 years (only 5,6,7 etc; there are no fractional values like 2.6 years) for a health variable which is non-negative, ...
2
votes
2answers
5k views

Multiple regression analysis - using all possible interactions

I have data on about 8000 persons and I am trying to find independent predictors of a health outcome variable (yvar). The predictor variables are age, gender, height, city and 3 other predictor ...
1
vote
3answers
359 views

'Punishment Function' in Number of Knots in Splines?

I was considering using natural cubic splines for my prediction problem when I had a thought: In Ridge Regression, you set out to minimize the equation; \begin{equation} F(X)=\lambda\sum_i ( b^2)+ \...
1
vote
1answer
964 views

Model Selection with Competing risks in Cox regression

When doing cox proportional hazards regression one often has competing risks. The typical approach for this is to fit separate cox proportional hazards models for each risk, censoring the competing ...
3
votes
1answer
228 views

Nature of the Relationship between Predictors and Dependent in Regression

Given the interpretation of regression coefficients for continuous predictors is of the form: a one unit increase in the predictor leads to a "coefficient" unit increase in the: dependent (linear ...
1
vote
1answer
62 views

Splines and clinical dataset query

Perhaps off topic or not directly suitable for crossvalidated but does anyone know of a good dataset with a clinical aspect I could use to teach myself how to use regression splines (in R)? I always ...
10
votes
2answers
1k views

How should I model interactions between explanatory variables when one of them may have quadratic and cubic terms?

I'm sincerely hoping that I have phrased this question in such a way that it can be definitively answered--if not, please let me know and I will try again! I should also I guess note that I will be ...
2
votes
3answers
2k views

What to do when some categories have too few observations

I have an ordinal, categorical variable with five levels, of which the last two have only one observation for each. Should I leave them alone, omit them, incorporate them in another category, or do ...
3
votes
1answer
10k views

Variable Selection for Logistic regression

I am performing logistic regression. I understand assumptions of logistic regression - Outliers, Multicollinearity. What i didn't understand how to select variables at beginning of model preparation. ...
0
votes
0answers
76 views

Quantifying the predictive ability of a model developed from a huge data set? (variation of bootstrapping?)

I have a statistical model with around 20 predictor variables, built on 90% of a dataset consisting of over 600k observations. The original developer held out 10% of the original dataset for the ...
14
votes
1answer
14k views

Logistic Regression with regression splines in R

I have been developing a logistic regression model based on retrospective data from a national trauma database of head injury in the UK. The key outcome is 30 day mortality (denoted as "Survive&...
5
votes
1answer
2k views

test interactions for multiple regression with many predictor variables

I have a data set with around 25 predictor variables. If I am planning to build multi-regression model against this data set. What are the general approaches to test the interactions of these ...
0
votes
1answer
966 views

Is it acceptable to transform data for use in a GLM using Poisson? [duplicate]

I have transformed my explanatory variables to a normal distribution as these variables include, proportions (logit transformed) and non normally distributed data (various transformations). The ...
11
votes
1answer
1k views

How to interpret variables that are excluded from or included in the lasso model?

I got from other posts that one cannot attribute 'importance' or 'significance' to predictor variables that enter a lasso model because calculating those variables' p-values or standard deviations is ...
18
votes
3answers
5k views

Model building and selection using Hosmer et al. 2013. Applied Logistic Regression in R

This is my first post on StackExchange, but I have been using it as a resource for quite a while, I will do my best to use the appropriate format and make the appropriate edits. Also, this is a multi-...
0
votes
0answers
533 views

Propensity score matching by gender in stata

Hello Stack Community! I'm trying to do a matching in Stata. I need to do a propensity score matching between a single individuals dataset (men and women) to a couple dataset. What I like to do is to ...
3
votes
1answer
2k views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be non-...
9
votes
4answers
4k views

Adjust for everything you have in propensity score?

I have a methodological question, and therefore no sample dataset is attached. I'm planning to do a propensity score adjusted Cox regression that aims to examine whether a certain drug will reduce ...
4
votes
1answer
593 views

Create age and sexmatched pairs to balance Cox regression further (updated)

I analyze ethnic differences in risk of cardiovascular events (CVD) in a cohort study of patients with coronary heart disease. It is known that immigrants have higher risk of CVD and I intend to show ...
0
votes
1answer
1k views

How to run main effects and interactions in a stepwise regression?

I am using multiple regression with the backward elimination method. I have one control variable (social desirable responding) and four predictor variables (gender and three self-esteem constructs). ...
1
vote
2answers
678 views

Interpreting ordinal regression result and calculating individual percentage contributions of independent variables

I ran an ordinal logistic regression in R using the polr function on a survey analysis dataset. The responses of the dependent variable range from Poor to Excellent. The responses to the independent ...
2
votes
1answer
721 views

Binary regression accuracy vs model fit in R

I ran two logistic regression models, one with a dataset including outliers and one without outliers, with multiple predictors. I checked each model's fit with the le Cessie – van Houwelingen – Copas ...
1
vote
2answers
146 views

Do you need to change instances to rates for OLS regressions

I am interested in performing a regression on data on a population. This population causes events X and Y. I have monthly data for population, event X, and event Y. Do I need to change my variables X ...
1
vote
1answer
936 views

Binary Logistic Regression Methods

I have data sample size of almost 15,000 cases. The dependent variable is a dichotomous variable stating whether the patient has the disease or not, Yes=1, and No=0. I have 12 more independent ...
2
votes
1answer
484 views

Using empirical null distribution to adjust odds ratios

I am doing a case-control study analysis with 2500 cases and 2500 controls. I am interested in finding out if the cases have higher odds of having a particular disease than the controls, so I am ...
3
votes
0answers
960 views

Catagorical variables with very uneven distributions? Removal/modify/leave?

In my current dataset I have quite a few categorical variables. Most have decent distributions between the categories. 30:40:30 splits etc. where these are percentage of dataset members per category ...
2
votes
2answers
2k views

High model fit but no significant impact of any of the predictors

I am applying binary logistic regression as my dependent variable is a dichotomous variable with 740 sample size. I have used enter method to input my variables and have designed two blocks. In block ...
8
votes
6answers
3k views

Dealing with non-normal distribution in "big" datasets, when do we throw out the CLT?

Apologies from the go as this question comes from an absolute newbie and will definitely not satisfy a lot of the detail required. Hence, your guidance in providing you the right information to allow ...
11
votes
3answers
4k views

GLM with continuous data piled up at zero

I am trying to run a model to estimate how well catastrophic illnesses such as TB, AIDS etc affect spending on hospitalization. I have "per hospitalization cost" as the dependent variable and various ...
1
vote
1answer
359 views

Confidence intervals for predictors in multivariate logistic regression

I've got a question. I am dealing with medical data which contain 5 predictors and 1 binary outcome. When I try to classify the data using all 5 predictors I get 0.84 area under roc-curve which is ...
8
votes
3answers
5k views

Internal validation via bootstrap: What ROC curve to present?

I am using the bootstrap approach for internal validation of a multivariate model built with either standard logistic regression OR elastic net. The procedure I use is as follows: 1) build model ...