0
votes
0answers
21 views

Generate mixture model from data with features

I want to build a mixture model from my data, but using features of my data to calculate each component in the model. The data: For each point I have 34 associated features. Each feature is a boolean ...
0
votes
0answers
19 views

Determine which variable or variables is/are the most efficient to predict the outcome

I have a small dataset (n=74) with a +/- 50 variables, not the best data but I have to work with it. The variables are used to select a product. I want to determine which variable or variables is/are ...
0
votes
0answers
24 views

How to deal with categorical features.

Recently I am playing in the famous Big-Data website Kaggle. There is a Display Advertising Challenge. In this competition, you are giving a training file which include huge records. the records is ...
2
votes
2answers
60 views

What interactions to include in my GLM model?

I realize this might be a too general question, so I'll describe what I'm doing right now first. I'm working for a virtual insurance company and I have this dataset. It has severity (meaning ...
5
votes
1answer
167 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
0
votes
1answer
143 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
1
vote
1answer
59 views

finding an optimal subgroup of binary indicators

My dependent variable is continuous variable that measures the (potential) success of a person in some activity. I have hundreds of binary indicators, each indicates about the existence of a specific ...
5
votes
1answer
160 views

A Bayesian perspective on omitted-variable bias (and other covariate-selection bias problems)

As I know OVB, from a frequentist education, when you leave a variable $(z)$ out of your control set $(X)$ that is correlated with both your independent variable of interest (treatment $T$) and your ...
0
votes
1answer
464 views

AIC, BIC, DIC, model selection criteria

I am trying to understand the difference between these parameters, and their application. Was hoping to get some correction/clarification to my statements. I have a training set and cross-validation ...
1
vote
2answers
149 views

BMI at baseline & followup with exposure at baseline; model change or BMI at FUP? Control for BMI baseline?

For a prospective occupational cohort where everyone is exposed to one or more chemical agents, examining BMI at follow-up compared to a specific chemical exposure at baseline, is it necessary to ...
2
votes
1answer
203 views

How to perform step() when n < p in R?

I am trying to perform stepwise regression for variable selection in R. In matlab, the stepwisefit function is able to work in ...
1
vote
0answers
41 views

How do I know if the differences in ICs among candidate models are significant?

I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question. My strategy is to develop a model with a subset of ...
0
votes
1answer
143 views

Partial correlation

I want to create a regression model to predict state crime rate. There are two variables among 10 ( Vi= # of violent crimes per 100,000 population, Vi2 = # of violent crimes per 10,000 population) ...
5
votes
3answers
356 views

Can I perform an exhaustive search with cross-validation for feature selection?

I have been reading some of the posts about feature selection and cross-validation but I still have questions about the correct procedure. Suppose I have a dataset with 10 features and I want to ...
4
votes
0answers
155 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
1answer
720 views

Variable selection with LASSO

I am trying to fit a predictive gene-based model in survival analysis. My question is: Can I use LASSO as a variable selection method, and then run a multivariate Cox regression to get the ...
3
votes
2answers
354 views

Variable analysis in multiple linear regression

I'm investigating how some weather variables (15) affect electricity demand in a specific area during the last 20 years. I was thinking to perform the following steps: 1. Perform Multiple Linear ...
13
votes
2answers
695 views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
3
votes
1answer
606 views

How to select the final model with elastic net feature selection, cross validation and SVM?

I have a dataset of some 100 samples, each with >10,000 features, some of which highly correlated. Here's what I am doing currently. Split the data set into three folds. For each fold, 2.1 Run ...
3
votes
1answer
391 views

Reference for random forests

I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests? Thanks. Below are two illustrative examples of why ...
11
votes
2answers
871 views

Best approach for model selection Bayesian or cross-validation?

When trying to select among various models or the number of features to include for, say prediction I can think of two approaches. Split the data into training and test sets. Better still, use ...
7
votes
3answers
1k views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all ...