1
vote
0answers
10 views

How to assess the importance of the features which come from intersection of features of the two models?

I have two models from two different data sets. Model 1 contain 50 features and model 2 contain 40 features. the intersection of features of model 1 and 2 is 10. so how can I assess the relative ...
1
vote
1answer
26 views

Why does SAS Enterprise Miner keep all dummy variables for a coded categorical variable in stepwise logistic regression?

SAS Enterprise Miner nicely creates coded dummy variables for any categorical variables used in a logistic regression model. When it performs a variable selection using stepwise sequential selection ...
1
vote
0answers
29 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
1
vote
0answers
27 views

Removing insignificant variables? [duplicate]

Suppose you fit a linear regression model on some data with 10 variables. The F-statistic shows that 3 of them are significant (p < 0.05) , 2 are within trend (0.05 < p < 0.10), and the other ...
2
votes
1answer
281 views

The main effect will be non-significant if the interaction is significant? [duplicate]

I am using linear mixed models to identify important factors, and it turns out that: A: significant B: not significant ...
0
votes
2answers
32 views

How to determine the factors correlated with observed data?

I have box-office collection data on a number of movies. I also have the production budget, director name, lead actor, actress, language and other meta data related to the movie. I want to know which ...
4
votes
3answers
487 views

Can independent variables with low correlation with dependent variable be significant predictors?

I have eight independent variables and one dependent. I have run a correlation matrix, and 5 of them have a low correlation with the DV. I have then run a stepwise multiple regression to see whether ...
0
votes
0answers
14 views

Variable Selection Methods in R [duplicate]

regsubsets and stepAIC are the two most common options for variable selection in R; they can be found in the ...
3
votes
2answers
296 views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
1
vote
0answers
36 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
3
votes
2answers
66 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
1
vote
0answers
27 views

Estimating confidence of a prediction

Given a set of features vectors $X=\{\vec{x}_1,..,\vec{x}_n\}$, binary ground truth data $Y=\{y_1,..,y_n\}$ and continuous prediction $\bar{Y} = \{\bar{y}_1,..,\bar{y}_n\}\in [0,1]$, I want to perform ...
1
vote
1answer
99 views

Using Principal Components Analysis for feature selection

I have a dataset D made of m samples, and n features with n >> m. For each sample I have a score s which I would like to ...
0
votes
0answers
25 views

Choice of 0 or -1 for failure in the independent variables of a logistic regression

I am performing some exploratory analysis on a dataset where the dependent variable is a dichotomous variable. I have ~10 explanatory variables, some of which are dichotomous observations. I am ...
0
votes
1answer
137 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
13
votes
2answers
548 views

Why does the Lasso provide Variable Selection?

I've been reading Elements of Statistical Learning, and I would like to know why the Lasso provides variable selection and ridge regression doesn't. Both methods minimize the residual sum of squares ...
0
votes
1answer
110 views

How Can I use some variables selected by LASSO?

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice. I have some data set. X = 500 x 100 (500 observations x 100 ...
1
vote
1answer
35 views

Interpretation of feature selection task

So I am given the following question Data set sample5.txt has a 20-dimensional input $x$ in $\mathbb{R}^{20}$ but we suspect that many of these are actually irrelevant. Could you model the ...
0
votes
0answers
64 views

LASSO method: prediction for multi-dimentional reponses

I have a feature matrix, that is 'X' 2000 (observation) x 200 (variable). I also have a response matrix, that is 'Y' 2000 (response) x 2 (variable). I would like to apply LASSO method to the data ...
1
vote
1answer
47 views

How to deal with features only available for a few instances?

I am working with a regression problem. I have some features which are only available for a few instances. But with those few instances based on those features we can build a model which gives ...
0
votes
1answer
241 views

Exploratory Factor Analysis for Binary Logistic Regression Variable Selection

I have a great interest in learning new methods(at least to me) of variable selection in regards to binary logistic regression when I am working with over 500 potential predictor variables and have ...
0
votes
1answer
79 views

Regression - Dealing with Correlated, Zero-Sum Predictors

I'm currently working on a regression problem where a subset of the predictor variables are zero-sum. By zero-sum I don't mean they all sum to zero, I simply mean that increasing one implies a ...
1
vote
1answer
99 views

Dependent variables in regression

I have two variables in a regression problem, where I predict the end interest rate for a loan application. x1: risk band (A+,A,B,C) x2: initial rate (6%, 7%, 8%, 9%) Based on these two variables, ...
1
vote
1answer
195 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
9
votes
3answers
327 views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
1
vote
0answers
90 views

True and false discovery rate in variable selection

I have a question about how I can calculate true and false positive rate in a simulation study? I have seen some articles and thesis by different definitions. One of them is the following one: ...
11
votes
2answers
458 views

Bayesian variable selection — does it really work?

I thought I might toy with some Bayesian variable selection, following a nice blog post and the linked papers therein. I wrote a program in rjags (where I am quite a rookie) and fetched price data ...
0
votes
1answer
139 views

Partial correlation

I want to create a regression model to predict state crime rate. There are two variables among 10 ( Vi= # of violent crimes per 100,000 population, Vi2 = # of violent crimes per 10,000 population) ...
3
votes
4answers
716 views

Decision Tree as variable selection for Logistic Regression

I have to do a Logistic Regression, and have to use a subset of the variables. I received this "tip": do a Decision Tree first, and use the most relevant variables in the Logistic Regression. Is this ...
-1
votes
1answer
155 views

Random forest like procedure for regression or other statistical models

I'm wondering if there exist methods similar to one used in random forest algorithm - I mean taking simultaneously bootstrap sample and random subset of features, then building statistisal model. Have ...
1
vote
1answer
237 views

How to define in R the most important variables?

I have a data set with my target gene and more than thousand transcriptional factors somehow correlated with this gene. There is data of these factors in more than 70 variable conditions. What I'm ...
3
votes
1answer
57 views

Methods for teasing apart the influence of different time series features on a target feature?

Are there any established methods for teasing apart the influence of different time series features on a target feature? To illustrate: The target: Sales volume of kittens. Features: Time of year, ...
3
votes
1answer
270 views

Why can't Bayesian variable selection be used with categorical variables with more than 2 levels?

I am reading this article which is the first approach on Bayesian variable selection. In the discussion section it says that one of the major limitations of the particular method is that it cannot be ...
6
votes
1answer
580 views

If p > n, the lasso selects at most n variables

One of the motivations for the elastic net was the following limitation of LASSO: "In the p > n case, the lasso selects at most n variables before it saturates, because of the nature of the convex ...
2
votes
1answer
257 views

Advice for a sparse high-dimensional regression strategy

I have a regression problem where I would like to predict values given several thousand sparse features. The general data set is an $n \times m$ matrix where each row contains a sample with a value I ...
4
votes
0answers
153 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
5
votes
1answer
808 views

Feature selection and parameter tuning with caret for random forest

I have data with a few thousand features and I want to do recursive feature selection (RFE) to remove uninformative ones. I do this with caret and RFE. However, I started thinking, if I want to get ...
17
votes
5answers
2k views

Detecting significant predictors out of 300 independent variables

In a dataset of two non-overlapping populations (patients & healthy, total $n=60$) I would like to find (out of $300$ independent variables) significant predictors for a continuous dependent ...
6
votes
1answer
2k views

What's the forward stagewise regression algorithm?

Maybe it's just that I'm tired, but I'm having trouble trying to understand the Forward Stagewise Regression algorithm. From "Elements of Statistical Learning" page 60: Forward-stagewise ...
12
votes
2answers
636 views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
1
vote
0answers
204 views

Guassian Process Regression - feature selection

I'm using guassian process regression to do some modeling. One issue I'm encountering is feature selection for some of my models, which often have many relevant features. I'm not sure what the best ...
8
votes
0answers
1k views

How do you select variables in a regression model?

The traditional approach to variable selection is to find variables that contribute the most to predicting a new response. Recently I learned of an alternative to this. In modeling variables that ...
2
votes
1answer
181 views

How to know which variables are more important in a process? [closed]

I have a process with 15 effective variables. I could record 9 variables to study its effect on process. I am looking for an appropriate factor to estimate the value of effectiveness of each factor. I ...
2
votes
2answers
68 views
2
votes
2answers
436 views

Variables importance: who can do the most pushups?

I don't know enough math to formulate an intelligent question on this so I'll give an example. I'd like an answer to my example but also I'd like to know the jargon I need to be able to research it ...
2
votes
0answers
460 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
9
votes
1answer
578 views

Random permutation test for feature selection

I am confused about permutation analysis for feature selection in a logistic regression context. Could you provide a clear explanation of the random permutation test and how does it applies to feature ...
18
votes
3answers
3k views

What are disadvantages of using the lasso for variable selection for regression?

From what I know, using lasso for variable selection handles the problem of correlated inputs. Also, since it is equivalent to Least Angle Regression, it is not slow computationally. However, many ...
7
votes
3answers
945 views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all ...
-1
votes
1answer
245 views

Shall I trust AIC (non-full model) or slope (full model)?

The purpose to run regressions for butterfly richness again 5 environmental variables is to show the importance rank of the independent variables mainly by AIC. In non-full models, they reveal that ...