Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
12 views

Fitting a good model in multiple linear regression cases

We assume to have multiple features and we would like to fit a good model through multiple regression. in the following case I can't graph all the features in a graph because we have many features and ...
0
votes
0answers
16 views

Contribution to the components of a Gaussian mixture by data features

My question is about modelling data with a GMM using EM. One can split the mean and variance of each component into parts as well when working with data with multiple features. My question is what ...
0
votes
0answers
7 views

Define feature for text classification using NLTK [on hold]

I'm working on Aspect Based sentiment analysis , I have a training set (text ,and aspectTerms) for each review. Using NLTK , I wan to build a NaiveBays Classifier that predict aspects of test unseen ...
0
votes
0answers
26 views

How to check that the selected features are not overfitting

I'm trying to select best features for sentiment classification for a set of reviews, and using penalized SVM and Logistic regression to perform such task. by basically iterating over different ...
0
votes
0answers
10 views

Performing interaction with a very significant predictor drives down p val of other predictors. But does it makes sense?

I'm observing a phenomenon that I can't understand. I have a linear regression setting with categorical vars. A couple of these elicit an highly significant coefficient and low p values. When used ...
1
vote
1answer
17 views

How to find the cause of defect in a process

Suppose a product A undergoes a certain process. This product A is produced at a rate of 8000 per month and out of those in 75 cases defects are generated. In the data set, I have rows corresponding ...
2
votes
1answer
38 views

Difference of variable selection and importance estimation

Isn't variable importance estimation a necessary prerequisite for variable selection? Is there any use case where you want to select non-important variables for your model? So, why is variable ...
0
votes
0answers
36 views

Multple linear regression, adding one predictor with almost perfect fit make others irrelevant

I found something interesting while playing with some data and linear regression. I built a regression with various predictors, more or less correlated with the outcome. Then I added one predictor ...
0
votes
1answer
22 views

Feature selection for unknown parametric model

Suppose one has about 500 points of 50 dimensional data that one knows a priori is derived from a parametric model (perhaps with some outliers). Does using this knowledge help in feature selection? I ...
2
votes
2answers
78 views

Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors?

I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face ...
0
votes
1answer
40 views

How to interpret random forest importance numbers

I ran randomForest in R package using 7 predictors variables (x1 to x7). I repeated the test with 4 dependent variables (y1 to y4). The importance numbers (IncNodePurity) are plotted in following ...
1
vote
2answers
71 views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
1
vote
0answers
10 views

Development data set for feature engineering and data exploration

I dont hear this being talked about much: If you want to engineer features and visually explore the data, should you do this on a development set separate from the training and test set? If ...
0
votes
0answers
24 views

Machine Learning: Potential Reasons of Precision Change after New Features are Added

My baseline model uses 10 features $[f_1, f_2, \dotsb, f_{10}]$. Now I have two new features $f_{11}$ and $f_{12}$. New models that use either $[f_1, f_2, \dotsb, f_{10}, f_{11}]$ or $[f_1, f_2, ...
0
votes
0answers
16 views

selection of features with Weka

I have a question and I hope that you can help me: I have a bilingual text(source language and target language). I extract from this text the best source phrase and the target phrase related to this ...
0
votes
1answer
27 views

Chi-squared Vs Mutual information

Is chi-squared feature selection better than Mutual information based feature selection mechanism?
0
votes
1answer
26 views

Effect of combining features on classification

I have 2 string features F1 and F2 based on which I am trying to perform classification. I have two choices, either to use the ...
0
votes
1answer
37 views

After Clustering, how can I evaluate which features had the biggest impact?

I've just performed unsupervised clustering (using DBSCAN) on a dataset for which I have no expert knowledge on. I'm interested in working out which features had the greatest impact on my clustering. ...
2
votes
2answers
79 views

Backward feature selection with CV model selection

I am thinking about doing the following to a data set with $N$ samples and $m$ features 1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best ...
0
votes
0answers
44 views

Advice for feature selection or feature extraction with semi-supervised learning

I am trying to solve a semi-supervised learning problem using LaplacianSVM. However, before applying LapSVM I would like either to perform feature selection or feature extraction. Furthermore, after ...
0
votes
0answers
8 views

Modeling 2-D data as vector

As part of my class project i had to use decision tree classification on a training set which contains a set of matrices where each row is a vector recorded at a particular time stamp and each row ...
0
votes
0answers
11 views

Can unsupervised feature learning be used to develop features reflecting patterns in human relationships?

Unsupervised feature learning has been used to learn features for objects and action classification and for emotion detection in speech. My questions are: (1) Is there any existing research ...
1
vote
1answer
29 views

Variable selection and validation dataset

According to Hastie & Tibshirani, we shouldn't use validation datasets to do variable selection; otherwise, we will overestimate the model fit. However, it seems quite often to select variables ...
0
votes
0answers
30 views

regarding using Lasso and Random forest based on the variable selection result coming from other processes

After the process of data exploration process and discussion with client, we set up a set of variables as follows: T1, T2, T3, T6, T8, T2*T3, T1*t6 During ...
0
votes
1answer
35 views

NLP tokenization for building feature vector

I am trying to match new product description with the existing ones. Product description looks like this: Panasonic DMC-FX07EB digital camera silver. These are steps to be performed: Tokenize ...
1
vote
1answer
57 views

How to handle missing data in a small $n$ large $k$ machine learning scenario?

I have a sample size $N=130$ and $1000$ variables. I am using machine learning techniques (SVM) for analysing the data. Some variables in the dataset have values that are so huge that they must be ...
1
vote
1answer
52 views
1
vote
0answers
15 views

Classification with two different dataset

I am working on a cancer classification model.Task is ,I am initially given a data set of 500 people and 1000 features.These people are given some kind of treatment(say Treatment 1). Some people are ...
1
vote
0answers
21 views

How features are representation in deep learning

I'm trying to build a deep neural network for mobile phone data. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
0
votes
0answers
5 views

selection variable algorithm based on conditonal mutual information

Is there anyone who understand the selection variable algorithm based on conditonal mutual information proposed by Fleuret 2004? http://sci2s.ugr.es/keel/pdf/specific/articulo/Fleuret_Fast_2004.pdf ...
1
vote
2answers
44 views

How to best to use Continuous value features with discreet values for logistic regression based binary classification problem

This is related to Minimisation algorithm for a mix of discreet and continuous parameters? I am trying out logistic regression to solve a binary classification problem. Though I am feature-scaling ...
0
votes
0answers
8 views

Non-linear auto-regressive model - preselection of relevant columns

Let us consider a dynamic system with nonlinear auto-regressive evolution such as $$ x_{t} = f(x_{t-1},x_{t-2},\dots,x_{t-d})+\epsilon_t $$ where $x_t\in\mathbb{R}^n$ is vector and $\epsilon_t$ is a ...
3
votes
1answer
62 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
1
vote
1answer
60 views

Grid search for SVM parameters; is this is really how it is done?

Suppose I use nested 10-fold cross-validation with SVM. So, the inner-most loop will go around 100 times. Now, suppose I use a gaussian radial basis kernel function, which needs the parameter sigma. ...
0
votes
0answers
11 views

Generating Labeled Training data from 2 data sources for Predictive Classifier

I am trying to build a predictive risk model classifier for an product (classifying good or bad). I am in the process of creating a training dataset. Here are the challenges I am facing. I have 2 ...
0
votes
0answers
18 views

Different variable importance results with stabsel and mboost

I'm using glmboost in the mboost package to fit a boosted regression using linear models as the base learner. There are 13200 observations and about 75 variables, ...
0
votes
0answers
18 views

Sparse PLS: algorithm for variable selection and model fitting

In the spls package in R (based on the manuscript by Chun and Keleş [1]), there is a separate specification for the variable selection and fitting in the main function, ...
0
votes
0answers
6 views

Low memory unsupervised feature selection

I have been working on a project that requires me to cluster data. I am currently using K-Means (The reason for the choice is a very long story, but basically I am stuck with it). So far I have ...
0
votes
0answers
10 views

Alghorithm for choosing the best set of words for twitter filtering

I'm using the twitter API to get a stream of tweets. You can't get all the tweets from the public API, it requires you to add some word filters. But you can add up to 400 words for filtering and if a ...
1
vote
2answers
75 views

How to do cross-validation when comparing different feature selection methods?

I am using SVM for a prediction task. My sample size is small, only N=140. Suppose I want to compare the prediction accuracy when using two different feature selection methods. Would it be better to: ...
4
votes
1answer
84 views

Relationship between Gini Importance and Prediction Performance (say AUC)?

I want to use the decrease in Gini impurity to rank features for my random forest classifier. I understand that the decrease in Gini impurity at one node is calculated as: $$ \Delta i(n) = i(n) - ...
1
vote
1answer
75 views

How to analyze elastic net fitted model coefficients

SOLVED: an elastic net model, as any other logistic regression model, will not generate more coefficients than input variables. Check Zach's answer to understand how from an (apparent) low number of ...
1
vote
1answer
34 views

Stratified sampling for creating test/training sets when there are continous and categorical variables to consider?

Assume a simple clinical study with N=200. Half of the participants are men and half of the participants are women. The hemoglobin of the participants ranges between 80 and 150. There's also several ...
1
vote
2answers
58 views

Can we learn 3d features using Autoencoder?

Typically, we use Autoencoder to learn 2d features on 2d images (e.g. pen-strokes of digit). For example, if I have 10000 3d 31x31x31 images (e.g. car images). I unroll each of the images, i.e. ...
1
vote
0answers
20 views

Regression to chose questions which better correlate with a 10 points likert like score

We have a survey with several questions with 5 likert scale points and we would like to compare the answers to those of another likert like question with 10 points. The approach we thought of is a ...
0
votes
0answers
18 views

how could i handle with missing data or non existent data? [duplicate]

i tried a forecasting method and i want to check if it is correct or not and why? my study is about evaluating mutual funds for two kind of them it is a comparative study and i wan to use gcc index ...
1
vote
0answers
29 views

Using t-test for feature selection after z-scoring data?

Suppose I have a high-dimensional dataset, and a binary classification problem. I want to use the two-sample t-test for feature selection. If the data has been normalized by z-scoring (so it has zero ...
2
votes
1answer
41 views

How to adjust data to remove influence of one or more features

For my first real data science project I would like to develop a model which better reflects review quality than "useful" votes. I am working with Yelp's latest Academic data set but this thinking ...
0
votes
0answers
27 views

Multicollinearity, feature selection for discriminant analysis and the error rate

I have a question regarding feature selection in LDA/QDA and deciding to eliminate variables to find an optimal model (lowest misclassification rate) I'm looking at how quadratic and linear ...
1
vote
1answer
118 views

Removing collinear variables for LDA/QDA in R

I'm new to R and I've been searching for a while for a function which can reduce the number of explanatory variables in my lda function (linear discriminant analysis). Basically, I've loaded the ...