2
votes
1answer
148 views

The main effect will be non-significant if the interaction is significant? [duplicate]

I am using linear mixed models to identify important factors, and it turns out that: A: significant B: not significant ...
0
votes
2answers
24 views

How to determine the factors correlated with observed data?

I have box-office collection data on a number of movies. I also have the production budget, director name, lead actor, actress, language and other meta data related to the movie. I want to know which ...
4
votes
3answers
152 views

Can independent variables with low correlation with dependent variable be significant predictors?

I have eight independent variables and one dependent. I have run a correlation matrix, and 5 of them have a low correlation with the DV. I have then run a stepwise multiple regression to see whether ...
0
votes
0answers
13 views

Variable Selection Methods in R [duplicate]

regsubsets and stepAIC are the two most common options for variable selection in R; they can be found in the ...
3
votes
2answers
222 views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
1
vote
0answers
32 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
3
votes
2answers
49 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
1
vote
0answers
25 views

Estimating confidence of a prediction

Given a set of features vectors $X=\{\vec{x}_1,..,\vec{x}_n\}$, binary ground truth data $Y=\{y_1,..,y_n\}$ and continuous prediction $\bar{Y} = \{\bar{y}_1,..,\bar{y}_n\}\in [0,1]$, I want to perform ...
1
vote
1answer
75 views

Using Principal Components Analysis for feature selection

I have a dataset D made of m samples, and n features with n >> m. For each sample I have a score s which I would like to ...
0
votes
0answers
24 views

Choice of 0 or -1 for failure in the independent variables of a logistic regression

I am performing some exploratory analysis on a dataset where the dependent variable is a dichotomous variable. I have ~10 explanatory variables, some of which are dichotomous observations. I am ...
0
votes
1answer
121 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
12
votes
2answers
395 views

Why does the Lasso provide Variable Selection?

I've been reading Elements of Statistical Learning, and I would like to know why the Lasso provides variable selection and ridge regression doesn't. Both methods minimize the residual sum of squares ...
0
votes
1answer
92 views

How Can I use some variables selected by LASSO?

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice. I have some data set. X = 500 x 100 (500 observations x 100 ...
1
vote
1answer
33 views

Interpretation of feature selection task

So I am given the following question Data set sample5.txt has a 20-dimensional input $x$ in $\mathbb{R}^{20}$ but we suspect that many of these are actually irrelevant. Could you model the ...
0
votes
0answers
59 views

LASSO method: prediction for multi-dimentional reponses

I have a feature matrix, that is 'X' 2000 (observation) x 200 (variable). I also have a response matrix, that is 'Y' 2000 (response) x 2 (variable). I would like to apply LASSO method to the data ...
1
vote
1answer
45 views

How to deal with features only available for a few instances?

I am working with a regression problem. I have some features which are only available for a few instances. But with those few instances based on those features we can build a model which gives ...
0
votes
1answer
185 views

Exploratory Factor Analysis for Binary Logistic Regression Variable Selection

I have a great interest in learning new methods(at least to me) of variable selection in regards to binary logistic regression when I am working with over 500 potential predictor variables and have ...
0
votes
1answer
71 views

Regression - Dealing with Correlated, Zero-Sum Predictors

I'm currently working on a regression problem where a subset of the predictor variables are zero-sum. By zero-sum I don't mean they all sum to zero, I simply mean that increasing one implies a ...
1
vote
1answer
93 views

Dependent variables in regression

I have two variables in a regression problem, where I predict the end interest rate for a loan application. x1: risk band (A+,A,B,C) x2: initial rate (6%, 7%, 8%, 9%) Based on these two variables, ...
1
vote
1answer
175 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
9
votes
3answers
300 views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
1
vote
0answers
82 views

True and false discovery rate in variable selection

I have a question about how I can calculate true and false positive rate in a simulation study? I have seen some articles and thesis by different definitions. One of them is the following one: ...
11
votes
2answers
445 views

Bayesian variable selection — does it really work?

I thought I might toy with some Bayesian variable selection, following a nice blog post and the linked papers therein. I wrote a program in rjags (where I am quite a rookie) and fetched price data ...
0
votes
1answer
123 views

Partial correlation

I want to create a regression model to predict state crime rate. There are two variables among 10 ( Vi= # of violent crimes per 100,000 population, Vi2 = # of violent crimes per 10,000 population) ...
3
votes
4answers
530 views

Decision Tree as variable selection for Logistic Regression

I have to do a Logistic Regression, and have to use a subset of the variables. I received this "tip": do a Decision Tree first, and use the most relevant variables in the Logistic Regression. Is this ...
-1
votes
1answer
150 views

Random forest like procedure for regression or other statistical models

I'm wondering if there exist methods similar to one used in random forest algorithm - I mean taking simultaneously bootstrap sample and random subset of features, then building statistisal model. Have ...
1
vote
1answer
226 views

How to define in R the most important variables?

I have a data set with my target gene and more than thousand transcriptional factors somehow correlated with this gene. There is data of these factors in more than 70 variable conditions. What I'm ...
3
votes
1answer
56 views

Methods for teasing apart the influence of different time series features on a target feature?

Are there any established methods for teasing apart the influence of different time series features on a target feature? To illustrate: The target: Sales volume of kittens. Features: Time of year, ...
3
votes
1answer
257 views

Why can't Bayesian variable selection be used with categorical variables with more than 2 levels?

I am reading this article which is the first approach on Bayesian variable selection. In the discussion section it says that one of the major limitations of the particular method is that it cannot be ...
6
votes
1answer
485 views

If p > n, the lasso selects at most n variables

One of the motivations for the elastic net was the following limitation of LASSO: "In the p > n case, the lasso selects at most n variables before it saturates, because of the nature of the convex ...
2
votes
1answer
225 views

Advice for a sparse high-dimensional regression strategy

I have a regression problem where I would like to predict values given several thousand sparse features. The general data set is an $n \times m$ matrix where each row contains a sample with a value I ...
4
votes
0answers
151 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
5
votes
1answer
690 views

Feature selection and parameter tuning with caret for random forest

I have data with a few thousand features and I want to do recursive feature selection (RFE) to remove uninformative ones. I do this with caret and RFE. However, I started thinking, if I want to get ...
17
votes
5answers
2k views

Detecting significant predictors out of 300 independent variables

In a dataset of two non-overlapping populations (patients & healthy, total $n=60$) I would like to find (out of $300$ independent variables) significant predictors for a continuous dependent ...
6
votes
1answer
2k views

What's the forward stagewise regression algorithm?

Maybe it's just that I'm tired, but I'm having trouble trying to understand the Forward Stagewise Regression algorithm. From "Elements of Statistical Learning" page 60: Forward-stagewise ...
11
votes
2answers
549 views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
1
vote
0answers
179 views

Guassian Process Regression - feature selection

I'm using guassian process regression to do some modeling. One issue I'm encountering is feature selection for some of my models, which often have many relevant features. I'm not sure what the best ...
8
votes
0answers
1k views

How do you select variables in a regression model?

The traditional approach to variable selection is to find variables that contribute the most to predicting a new response. Recently I learned of an alternative to this. In modeling variables that ...
2
votes
1answer
179 views

How to know which variables are more important in a process? [closed]

I have a process with 15 effective variables. I could record 9 variables to study its effect on process. I am looking for an appropriate factor to estimate the value of effectiveness of each factor. I ...
2
votes
2answers
64 views
2
votes
2answers
427 views

Variables importance: who can do the most pushups?

I don't know enough math to formulate an intelligent question on this so I'll give an example. I'd like an answer to my example but also I'd like to know the jargon I need to be able to research it ...
2
votes
0answers
420 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
9
votes
1answer
539 views

Random permutation test for feature selection

I am confused about permutation analysis for feature selection in a logistic regression context. Could you provide a clear explanation of the random permutation test and how does it applies to feature ...
15
votes
3answers
3k views

What are disadvantages of using the lasso for variable selection for regression?

From what I know, using lasso for variable selection handles the problem of correlated inputs. Also, since it is equivalent to Least Angle Regression, it is not slow computationally. However, many ...
7
votes
2answers
874 views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all ...
-1
votes
1answer
245 views

Shall I trust AIC (non-full model) or slope (full model)?

The purpose to run regressions for butterfly richness again 5 environmental variables is to show the importance rank of the independent variables mainly by AIC. In non-full models, they reveal that ...