New answers tagged

0

I think a greedy approach would make most sense. I.e., first find the variable that is most highly correlated to the target variable. Then, for each additional variable, you need to find some objective with regard to weighting the correlation to the previously selected variable or the target variable is more important. You may want to look into sequential ...


0

Good question because, although you have to notice a couple of things that I will mention below, it is a good point for many real-life applications. First of all, just to be precise, when you say “In building the model, I think it makes sense to try to capture the individual effects of, and interactions between, my predictors.”, you have to notice that when ...


1

For correlations between continuous and categorical variables see Correlations between continuous and categorical (nominal) variables and Correlations with unordered categorical variables. But your main question seems to be about classification into two classes, since the target is binary. It is in most cases better to see this as a risk estimation ...


1

If you set the column (in the data frame you give to h2o.predict) to all NA, then it will act as if the data is missing. According to the FAQ (and assuming you had no NAs in the training data) it will then "follow the majority direction (the direction with the most observations)". Compared to rebuilding the model without the variable, I would expect this to ...


0

Try mutual_info_classif scoring function. It works with both continuous and discrete variables. You can specify a mask or indices of discrete features in discrete_features parameter: >>> from functools import partial >>> from sklearn.feature_selection import mutual_info_classif, SelectKBest >>> discrete_feat_idx = [1, 3] # an ...


1

@astel's advise is spot on: which features to use is part of the hyperparameters your model has, thus Outer loop: Model evaluation Inner loop: hyperparameter optimization, including feature selection is the way to go. The important reason behind that is that feature selection and optimization of other hyperparameters are usually not independent, and ...


0

Current experiments suggest LR is indeed susceptible to such feature re-weightings. Further, there is the trivial case when a given $w = 0$, as here no signal is present.


0

None of the methods you mention are used for feature selection. You have high-dimensional data (where $p>n$), so regular methods go out the window. Given your data, and the fact that you want to keep your variables intact and reduce the dimensionality, then something like LASSO is more appropriate, or a Random Forest.


0

The problem, as stated, is a little weird. You say your sample has classes A, B and others. But you are unsure about the others as they may belong in A, B or something else completely? You need to decide what you are going to do here, are you going to assume the classes are known and do supervised learning? Are you going to assume the classes are not really ...


1

Let $X1 = 1$ if "yes" and $0$ if "no". Then, you can impute the value $X2 = 0$ for the units of observation having $X1=0$, and include in your regression the interaction term $(X1\cdot X2)$ instead of $X2$, so that the effect of $X2$ is conditional on $X1 = 1$.


5

This sounds like a clique detection problem. To get there, we will frame the problem as one of graph theory. First, set all elements above your threshold to 1 and all other elements to 0. This is a symmetric adjacency matrix. In your example, the subsets are all either cliques or singletons. A clique is a complete subgraph, so the task reduces to ...


1

To me, this question sounds like a constrained, dynamic, real-time dimension reduction problem. There are many possible methods which would address this and the choice is a matter for subjective judgement, predisposition, skill and training. Basing that choice on pairwise correlations alone would not be on my list of options. PCA (principal components ...


1

I don't think either is more or less relevant or robust for five-point Likert-type scales. The key consideration is more focused on whether any model selection has occurred. Overall, the two methods are fairly different in terms of how they approach importance but can result in similar results as is discussed by Groemping here. The Shapley/Dominance/LMG ...


2

Haven't got enough reputation to comment, so have to answer here. Have you tried SHAP values? Rather easy to implement, and gives really intuitive answers. You can find a quick example with XGBoost here. Alternatively, there's LIME, however I prefer SHAP myself.


0

Let's first sort things a bit: train_k_features = function (train, k){ run f-test on train data. rank features according to their f-scores. apply k-best algorithm with different *k* values (previously sampled from uniform random distribution) fit model on train data with k-best features, and test on test data. return model } with ...


3

I was able to come up with a solution to this problem, and wanted to post here in case anyone else found it helpful. Comments by usεr11852 about sparse encoding helped point me in the right direction. My strategy was as follows. 1) Restructure my SQL pull in a long format, to avoid the column issue I was facing in SQL. See example below: Patient_ID,Var,Val ...


0

@whuber 50 million comes when you have 100*100 pixel image. where square(100*100) = 100000000 (10 million) and square(100*100)/2 = 5 million. Hope this answers.


1

Until you're ready to train your full model on all data, you should treat your out-of-sample data sets as nonexistent when you're doing work for the in-sample data. If you have 1000 observations split into 5 sets of 200 for 5-fold CV, you pretend like one of the folds doesn't exist when you work on the remaining 800 observations. If you want to run PCA, for ...


0

I would start with the most simple possible solution, and only if that doesn't work I'd start to look into more complicated options. The first place to start when you wish to reduce the number of variables in your data is principal components analysis (PCA). This looks for new features that are linear combinations of your old features, and summarize your ...


0

You say the goal is to predict price of hotels. Why not start out with that, maybe with regularized regression? Why do feature selection at all? See Variable selection for predictive modeling really needed in 2016? and What kind of feature selection can Chi square test be used for?. So, go for regularized regression followed by validation. Maybe some ideas ...


2

Maybe LASSO is right and none of the variables are good predictors of your outcome (not about significance). If you want a less stringent form of LASSO you can use the ELASTIC NET. RFE is not a good approach for feature selection. You should compare different models with predictions from a test set, to check whether they truly learned anything.


Top 50 recent answers are included