# Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

1,348 questions
Filter by
Sorted by
Tagged with
748 views

### Confidence Interval - Binary classification [duplicate]

How do we calculate a confidence interval for a result in binary classifiers ? CI for regression problems makes sense since we have a variable estimated output that I can calculate its estimated mean ...
423 views

### What characteristics should the input data have for a neural network?

I am planning to use a neural network for prediction. For example, to predict whether a student will pass a course based on his previous academic records or characteristics. I was wondering how to ...
153 views

### Concluding from Learning and validation curves

Background I am fitting a dataset of 1500 observations and 375 features (after one hot encoding of categorical features) dealing with prediction of house prices. I am using a gradient boosting model (...
87 views

### Fit a function f on dataset X such that f(X) fits a histogram

I have dataset $X=\{\boldsymbol{x_1},\boldsymbol{x_2},\dots,\boldsymbol{x_n}\}$ and $Y=\{y_1,y_2,\dots,y_n\}$ and want to learn a function $f$ such that $y = f(\boldsymbol{x})$ can be approximated as ...
432 views

### Difference between semi-supervised learning and prediction?

What is the difference between semi-supervised learning and prediction? It seems to me they're the same (both are predicting the label)
603 views

### How to represent outliers for multi dimensional data (local outlier factor)

Below graph taken from http://en.wikipedia.org/wiki/Local_outlier_factor displays "LOF scores : LOF image : This is great for two dimensional data but what about data > than two dimensions. How ...
487 views

### Xgboost Feature Importance shift

If I plot the feature importance of my xgboost model I get for example f10,f3,f7,f99,... as the most important features. Now I decided to remove f3 and I imagined the new feature importance would be ...
1k views

### How does regularization work for a Gaussian Process classification model?

I'm a bit confused about Gaussian Process models for classification. In chapter 3 of http://www.gaussianprocess.org/gpml/ it is claimed that you can use a logit or probit model without any additional ...
813 views

### How should I use Recurrent Neural Network to model this problem? [closed]

I am using Keras to do a machine learning task: Let's say I want to predict the time that a user spends on a product page. Each training case is a partial user visit session. One single user may ...
114 views

### What cause $X\beta$ shift from Stochastic Gradient Descent Comparing to Logistic Regression?

I am experimenting with stochastic gradient descent and observing very strange output. In a toy problem, the $X\beta$ for stochastic gradient descent is always larger than $0$, which will be ...
1k views

### Creating a Predictive Model with Binned Data

I have a health dataset with the number of drinks per month someone consumes, and many other variables that are binned. For example, 1: income less than \$10000, 2=income less than \$20000, and so on. ...
2k views

### How to normalize filters in convolutional neural networks?

Usually, when convolving images, the elements in the filter sum to one. Is this criterion enforced in convolutional neural networks? If yes, how?
223 views

### Maximum Likelihood Estimate (MLE) equivalent to finding $\hat y$ in linear regression with i.i.d. Gaussian noise distribution

In an assignment I need to show that for linear regression, with the noise i.i.d. Gaussian distributed $\epsilon_i \sim N(0,\sigma^2)$, that finding the Maximum Likelihood Estimate (MLE) is equivalent ...
469 views

### Which is the best classifier and with what performance measures?

I tried to implement a Classifier comparison like in the scikit-learn for text classification. I used an 81 instances as a training sample and a 46 instances as a test sample. I tried several ...
508 views

### How to add more inputs to a CNN?

What would be the correct approach to add additional inputs that aren't images, e.g. time, to the CNN. I initially thought of adding more inputs to one of the densely connected layers at the end of ...
136 views

### Changeing the hypothesis while generating samples

I'm currently reading / working through: Learning from Data: A short course by Abu-Mostafa et. al to familiarize my self with the shift in language from Stats to ML. In the section on feasibility we ...
102 views

### Trees of ensembles.

I have a large dataset (100k+), and it's growing everyday. I want to train it to predict a value (a regression problem). I've been finding that ensemble trees work the best for now, but in the ...
202 views

### Spearman rho statistical significance value (z)

How can I calculate the statistical significance (Z) of spearman's footrule rho? I came across the formula at this wiki page ...
102 views

### Getting critics to recognize that two similar input patterns refer to the same output-performance relationship

The actor-critic model is used within temporal difference learning, which is a method within reinforcement learning, to optimize a process on a state-by-state basis by using the difference between ...
607 views

### Find covariance if given mean and variance

I have a signal x that I want to classify in one of the classes A and B in which the means are Ma=[0.5,0.6] and Mb=[2,2] and with variances ...
156 views

### Recommendations for textbooks covering current data mining/machine learning techniques for fraud detection?

I work in the health insurance field, but a general treatment of fraud detection methodologies would still be helpful. So far I've discovered brief articles outlining particular techniques used in ...
101 views

### Scaling in linear regression

The text is from Intro to Statistical Learning Page no 380.Can anyone explain the both ideas clearly with an example if possible 1) In linear regression scaling has no effect. 2)In linear ...
187 views

### Result of K-Means Algorithm Not Desired

I am learning about K-means algorithm, and I have generated a dataset with 150000 data points, with 10000 points per cluster. (Scatter plot at the bottom) When I run K-means on the dataset, I first ...
2k views

### How to handle skewed binary target variables? [duplicate]

Possible Duplicate: Supervised learning with “rare” events, when rarity is due to the large number of counter-factual events I am trying to predict diabetes using the BRFSS dataset by ...
292 views

### Representation input and output nodes in neural network for $\textit{AlphaZero}$ chess?

I am wondering how the neural network for AlphaZero chess works. I know that it takes a historic set of states of the board as input nodes. But I am wondering how many output nodes there are and what ...
345 views

### Using ML approaches to build a recommender engine for sales team

I work at a startup as a developer, but I wanted to help out our sales team with running some ML algorithms on the data. A bit of context: Most of our revenue comes from ad purchases, so in a nutshell,...
223 views

### Imbalanced Test Data

I have an imbalanced (1:5) training and test set with only two classes and have oversampled the training set with SMOTE so that the class ratio is 1:1. The ML model gives values over 0.7 for accuracy, ...
169 views

### Tuning priors/weights/costs to counteract class imbalance

I have a classification problem which consists of two classes. It has high class imbalance. There are around 85% data points for the negative class and only 15% for the positive class. One option is ...
1k views

### Pre-processing (center, scale, impute) among training sets (different forms) and the test set - what is a good approach?

I am currently working on a multi-class classification problem with a large training set. However, it has some specific characteristics, which induced me to experiment with it, resulting in few ...
147 views

### Sample space for hypothseis, training data of bayes theorem

I am learning about Bayes theorem in machine learning . $p(h/D) = \frac{p(D/h)p(h)}{p(D)}$ $p(h) =$prior probability of hypothesis h $p(D)$ = prior probability of training data D $p(h/D)$ = ...
363 views

### Should difference between accuracy of model on training data and testing data be considered for model selection?

Suppose I have two models (Model 1 and Model 2), Where Accuracy of Model 1 on test data is higher than that in Model 2 Difference between accuracy of model on test data and training data is higher ...
331 views

### How may I convert Perplexity to F Measure

In the practice of Machine Learning accuracy of some models are determined by perplexity, (like LDA), while many of them (Naive Bayes, HMM,etc..) by F Measure. I like to evaluate all the models with ...
125 views

### Retrain random forest with important variables

So I have a classification problem with around 2000 predictors. First I run a random forest model to get important variables. Then I only use those variables (let say the top 30) to run the model ...
798 views

### Sparse Additive Generative Models (SAGE) v/s LDA (Latent Dirichlet Allocation)

Intuitively, how are Sparse Additive Generate Models (SAGE) by Eisenstein, different from Multinomial Dirichlet distributions. I understood that in this model distributions are added in logarithmic ...
82 views

### Which ML algorithm is suitable for the following problem?

Assume i have x states each defined by an n-dimensional feature vector. In addition i have a set of actions which can be taken in each state resulting in a state action score. What would be an ...
41 views

### How to use random forest for regression after it is trained

I don't understand how to work with a random forest regressor after it is trained. I read and coded some tutorials about regression with random forests in Python with scikit but I don't understand how ...
220 views

### Misunderstanding of E-Greedy Monte Carlo Proof

I'm confused about one step of the e-greedy Monte Carlo control proof on page 83 of Sutton and Barto Reinforcement Learning. The book annotates saying "(the sum is a weighted average with ...
2k views

### out of bag error in random forest and data partitioning

I have a question concerning OOB error in random forests and data partitioning. As far as i know in random forests the trees are not pruned. Also we use OOB error for measuring the performance of the ...
278 views

### Question about implementing nested Chinese Restaurant Process (nCRP)

I am trying to follow the original paper on nCRP by Blei et al., 2010 and am confused with it's implementation. The authors describe the analogy for an nCRP as follows: A tourist arrives at the ...
3k views

### Different results from several “passes” of Random Forest on same dataset

I've been playing around with the German Credit dataset available in Kuhn & Johnson's caret package for ...
426 views

### Should I use the same weight initialization for each fold in cross validation?

Say, for example, I have 5 splits of my data. Can I randomly initialize the weights for my neural network at the start of each split? Or should I save the initial weights randomly initialized for the ...
66 views

### How can I understand the concept of a noise in machine learning?

In Bishop's book, one of the first examples is shown here Essentially, the data $x$ are randomly generated, and $t$ are generated by running $x$ through a function $\sin(2\pi x)$, then Gaussian ...
179 views

### How does one show that the multi-class hinge loss upper bounds the 1-0 loss?

I was trying to understand the mathematical claim that: $$\mathbb{1}\{ \hat y \neq y \} \leq \max_{j \in \{1,...,K \} } \left( \mathbb{1}\{ i \neq y\} + W^T_jx - W^T_y x \right)$$ I tried showing ...
2k views

### Activation functions for autoencoder performing regression

I want to train both a single-layer autoencoder and a multi-layer autoencdoer in Keras to reconstruct an input with 24 features, all in the same scale with int values from 0 to ~200000. My question is:...
33 views

### Expectation of matrix X multiply by indicator matrix

Let X1, . . . , Xn, X be i.i.d. R-valued random variables Suppose an indicator matrix I{A} be 1 if A is true and 0 otherwise. Then for τ > 0, and How this could be true....? Can anyone walk ...
243 views

### Does Matching Pursuit and Soft Thresholding return the same minimizer?

I wanted to understand if the solutions (minimizers) obtained by Matching Pursuit algorithms (say Basis Pursuit denoising) and Soft Thresholding yielded the same minimizer (same solution or same ...
82 views

### Simple prediction example

Given Training data: x y output 0 0 0 1 0 1 1 1 1 Predict output for Testing data: x y output 0 1 ? The particular input variable combination was not seen before. So, would you ...
2k views

### Large data variable selection

I'm looking for some methods of variable selection on large datasets.The number of variables are around 30-40, but the number of observations is quite large (around 36000000) Any methods which I ...