4
votes
1answer
85 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
0
votes
1answer
121 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
1
vote
1answer
47 views

finding an optimal subgroup of binary indicators

My dependent variable is continuous variable that measures the (potential) success of a person in some activity. I have hundreds of binary indicators, each indicates about the existence of a specific ...
5
votes
1answer
119 views

A Bayesian perspective on omitted-variable bias (and other covariate-selection bias problems)

As I know OVB, from a frequentist education, when you leave a variable $(z)$ out of your control set $(X)$ that is correlated with both your independent variable of interest (treatment $T$) and your ...
0
votes
1answer
375 views

AIC, BIC, DIC, model selection criteria

I am trying to understand the difference between these parameters, and their application. Was hoping to get some correction/clarification to my statements. I have a training set and cross-validation ...
0
votes
0answers
33 views

Choosing prior distribution in LDA

how do you set prior distribution of K in LDA and can it be used for feature selection to improved selection accuracy of document. Abbey
1
vote
2answers
106 views

BMI at baseline & followup with exposure at baseline; model change or BMI at FUP? Control for BMI baseline?

For a prospective occupational cohort where everyone is exposed to one or more chemical agents, examining BMI at follow-up compared to a specific chemical exposure at baseline, is it necessary to ...
2
votes
1answer
163 views

How to perform step() when n < p in R?

I am trying to perform stepwise regression for variable selection in R. In matlab, the stepwisefit function is able to work in ...
1
vote
0answers
41 views

How do I know if the differences in ICs among candidate models are significant?

I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question. My strategy is to develop a model with a subset of ...
0
votes
1answer
124 views

Partial correlation

I want to create a regression model to predict state crime rate. There are two variables among 10 ( Vi= # of violent crimes per 100,000 population, Vi2 = # of violent crimes per 10,000 population) ...
5
votes
3answers
266 views

Can I perform an exhaustive search with cross-validation for feature selection?

I have been reading some of the posts about feature selection and cross-validation but I still have questions about the correct procedure. Suppose I have a dataset with 10 features and I want to ...
4
votes
0answers
151 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
1answer
574 views

Variable selection with LASSO

I am trying to fit a predictive gene-based model in survival analysis. My question is: Can I use LASSO as a variable selection method, and then run a multivariate Cox regression to get the ...
3
votes
2answers
308 views

Variable analysis in multiple linear regression

I'm investigating how some weather variables (15) affect electricity demand in a specific area during the last 20 years. I was thinking to perform the following steps: 1. Perform Multiple Linear ...
11
votes
2answers
551 views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
2
votes
1answer
500 views

How to select the final model with elastic net feature selection, cross validation and SVM?

I have a dataset of some 100 samples, each with >10,000 features, some of which highly correlated. Here's what I am doing currently. Split the data set into three folds. For each fold, 2.1 Run ...
3
votes
1answer
339 views

Reference for random forests

I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests? Thanks. Below are two illustrative examples of why ...
11
votes
2answers
746 views

Best approach for model selection Bayesian or cross-validation?

When trying to select among various models or the number of features to include for, say prediction I can think of two approaches. Split the data into training and test sets. Better still, use ...
7
votes
2answers
874 views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all ...