Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4,452 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
199 views

Why is bridge regression called “bridge”?

Bridge regression coefficient estimate $\hat{β}^{br}$ are the values that minimize the \begin{equation} \text{RSS} + \lambda \sum_{j=1}^p|\beta_j|^q , \end{equation} where $q \in \mathbb{R}$ and $q &...
3
votes
0answers
382 views

How to handle Zeros in dependent variable in Multiple Linear regression

I am totally new to machine learning (and to this platform too) and was trying to implement Multiple linear Regression to improve my ranking algorithm. I have a data-set which have the following ...
3
votes
0answers
176 views

Number of observations in a node in XGBoost

I understand how the cover is calculated in XGBoost, the sum hessian at that node. For the root node of tree 1 for binary logistic, it becomes n(.5)(1-.5) with base score as 0.5. The cover at root ...
3
votes
2answers
71 views

How to handle machine learning inputs that should be considered as set of vectors, but whoes interpretation is order invariant (order agnostic set)

Basically wondering best practices for input modeling and ML algorithm type(s) for inputs that essentially model samples that are a bag/set of "sub-objects", so order does not matter. Think of the ...
3
votes
0answers
223 views

General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$ where $C$ is the ...
3
votes
0answers
99 views

Why use separate trees for each class in multi-class gradient boosting?

Gradient boosted decision trees can be used to solve multi-class classification problems. Friedman (2001) fit $K$ trees on each iteration—one for each class. Multiple GBM implementations also follow ...
3
votes
0answers
57 views

Find Feature Weighting in Deep Learning

If I train a deep neural network on standard tabular data (csv file etc. with labeled features) is there a good way to gauge how important each feature is in a particular new instance's prediction ...
3
votes
0answers
204 views

Bias Variance decomposition derivation question/proof (from Wikipedia)

I have a question on this derivation of the bias-variance decomposition. At some point they have this part of the expression --> $\mathbf{E}[2y\hat{f}]$ and they say that $\epsilon$ and $\hat{f}$ are ...
3
votes
0answers
380 views

Multinomial Logistic Regreesion with Lasso penalty in R

I am applying regularized logistic regression (in R) to the handwritten digits data set. I have fitted a logistic multinomial model with lasso penalty to the training data. I am asked to obtain the ...
3
votes
1answer
349 views

Combining classification and anomaly detection

I want to build a system, that can classify known classes in a supervised way and at the same time tells if there is a new anomaly class it has not seen before. The user can then label that unknown ...
3
votes
1answer
102 views

Sampling a test set from global spatial data

The basis of testing the accuracy of any machine learning algorithm is to test the trained algorithm on data that it has never seen before. The usual approach to sample the test set is to just ...
3
votes
0answers
82 views

“Data beats hardware and algorithms in neural nets” paper?

I'm trying to track down the citation information for an article. The paper concerned itself with the recent explosion in successful applications of neural networks, and whether this was cause by ...
3
votes
1answer
62 views

Is it valid to calculate propensity score for each treated individual separately?

I have temporal twitter data, and I want to calculate propensity score for the treatment and control group. The problem is, the treatment happened at different time for different user, and I want to ...
3
votes
0answers
966 views

Are XGBoost probabilities well-calibrated?

In general, can you say anything about how well are the probabilities returned by XGBoost are calibrated? Is it true that, because XGBoost directly optimizes log-loss, probabilities are generally well-...
3
votes
0answers
250 views

Voting between classifiers : How to prove it works?

Assume m independent binary classifiers with probability $p$ to be correct $p>0.5$. Show that the probability of a voting, e.g. decision is made by the majority of classifiers is correct with ...
3
votes
0answers
1k views

Why do CNN's use ReLU over Sigmoid function?

I am trying to map my basic understanding of MLP's to CNN's. Why does a CNN sacrifice all negative inputs with the ReLU over the sigmoid. Is it because: The sigmoid has a range of between zero and 1 ...
3
votes
0answers
121 views

Assessing correlated predictions

Let's assume we have a prediction algorithm (if it helps, imagine it's using some boosted tree method) that does daily predictions for whether some event will happen to a unit (e.g. a machine that ...
3
votes
0answers
359 views

Position Bias Normalization in CTR Prediction

I am working on Click Through Rate(CTR) prediction model on a toy dataset. The label I am using is #Click/#NumShown. But there is position bias in results shown. ...
3
votes
0answers
343 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
3
votes
1answer
36 views

What is Generative Training?

I know that the difference between generative and discriminative classifiers is that the generative ones directly model the distribution of the observed data while the discriminative ones do not. ...
3
votes
0answers
89 views

State of the art feature extractor from text for machine learning

In machine learning for text classification, the first step after acquiring and cleaning the data is that of feature extraction. Since computers can't understand language like humans, the language ...
3
votes
0answers
382 views

Using confusion matrix to improve my SVM

I ran an SVM classifier on the CIFAR_10 classification workbench. I got about 2/3 accuracy (which I think is Ok, but I want to improve...) Here is my confusion matrix: ...
3
votes
0answers
85 views

Deriving bias of local linear regression

I have been reading Elements of Statistical Learning, and in Chapter 6 on Local Regression, they state the following for fitting local regression at point $x_0$ from data $\mathbf{x}$ of size $\rm{dim}...
3
votes
0answers
454 views

How to properly normalize multivariate time series data?

My data has three kinds of features, with different distributions and ranges, and I am going to use RNN to model such data. In the process of normalization, my initial thoughts are: compute z-score ...
3
votes
2answers
3k views

Simple Explanation of Baum Welch/Viterbi

I'm looking for a very simple explanation as possible for Baum Welch and Viterbi for HMMs with a straightforward well annotated example. Almost all of the explanations I find on the net invariably ...
3
votes
2answers
40 views

Is it problematic if most of the weights or biases in a hidden layer are the same sign?

I'm trying to diagnose overfitting in my multi-layer perceptron by looking at the weights, biases and gradients in each layer. I'm noticing that in the neural network that is overfitting, the weights ...
3
votes
0answers
686 views

When, if at all, to reset the state of an LSTM when training and when testing?

I am building an LSTM that takes in time-series financial data. My dataset is made up of IDs (each ID is a certain stock), and timestamps. For each ID at each timestamp, there are a number of features ...
3
votes
1answer
605 views

Keras: val_loss decreases while loss increases

I set up a model in keras (in python 2.7) to predict the next stock price in a particular sequence. The model I used is shown below (edited to fit this page): ...
3
votes
0answers
465 views

How does choosing between pre and post zero padding of sequences impact results

I'm working on an NLP sequence labelling problem. My data consists of variable length sequences (w_1, w_2, ..., w_k) with corresponding labels ...
3
votes
0answers
133 views

Model training: difference between inference, marginalisation, and estimation

I am working through the lecture slides of Carl Rasmussen's Probablistic Machine Learning course. In slide 6 of the first lecture it lists a number of ways that one can learn the parameters (A, C) and ...
3
votes
0answers
64 views

What are some machine learning problems that can be attacked with continuous multiobjective optimization?

I am working on continuous vector optimization, and hence continuous multiobjective optimization is a particular case. I am interested in finding applications in machine learning for this problems. Is ...
3
votes
2answers
2k views

Observations weight-age in a Machine Learning model

I want to know is there any way in R/Python to specify to the model to emphasize its learning more on specific subset of data , while it considers the whole data. For example - i have sales behavior ...
3
votes
0answers
226 views

Expectation of Covariance Matrix for Deep Gaussian Processes

I am currently reading the paper entitled "Variational Auto-Encoded Deep Gaussian Processes" by Dai et al, a copy of which may be found here. The paper proposes a stacking of Gaussian Process Latent ...
3
votes
0answers
292 views

Quiz: Determine first principal component from data-plots

We see four data plots. The goal: How does the first principal component look for each plot a-d. For plot d, it is true that both clusters have same number of datapoints. First principal component ...
3
votes
1answer
1k views

Backpropagation proof and usage confusion

I've been taking Andrew Ng's course on Coursera, and although it has been great so far, I loathe his lack of supplementary documents on proofs. Thankfully, there are some great articles found pretty ...
3
votes
1answer
1k views

Sequence lengths in LSTM / BiLSTMs and overfitting

I'm currently working with LSTMs and BiLSTMs, using Keras as library (TF backend). Following Tutorials and reading some papers, I found out that the sequences used are mostly quite short. What I do ...
3
votes
0answers
44 views

Should I select features for imputation

When imputing a missing value (in my case using MICE) - should I use all the variables in the dataset or should I use only the variable which correlate most with the missing values I want to impute? ...
3
votes
0answers
363 views

How to use MCMC / gibbs sampling instead of an optimization algorithm ?

I've tried and implementend Factorization Machines with different loss functions and optimization algorithms (SGD , coordinate descent, adagrad, adadelta ...) and I've seen that it's possible to use ...
3
votes
1answer
2k views

Optimization algorithms for sparse data

For couple of weeks now I've been dealing with a classification problem involving a sparse dataset. To be more specific, for each input $x^{(i)}$, knowing that I have 1000 features, I've only 5 to 10 ...
3
votes
0answers
571 views

Why do my Feature Importance and Partial Dependence plots not agree?

I need some help understanding my partial dependence plots for features passed to a GradientBoostClassifier when comparing them to the feature importances. For some background, my goal here is to ...
3
votes
0answers
188 views

Bias-variance: is it really a “trade-off”?

For some estimator $\hat{\theta}$, we have the "bias-variance trade-off": $MSE(\hat{\theta}) = bias^2(\hat{\theta}) + var(\hat{\theta}).$ When I think of a trade-off, I would expect that as the ...
3
votes
0answers
289 views

Interaction effect in random forest

I'm interested in interaction effect between variables in random forest. I found some information here https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#workings. The operating ...
3
votes
0answers
400 views

How does one actually apply SGD to word2vec models?

How does one actually apply SGD to word to vec? My question are: What entails a mini-batch? Do we (at least conceptually) take derivatives wrt to all model params? How does changing the objective ...
3
votes
0answers
198 views

closed form solution for the optimization problem

I have some what complex minimization problem, where the objective function is of the form, $$ \lambda\sum\limits_{i=1}^m\|X_i\|_2 + \nu\|X\|_1 + \frac{\rho}{2}\|XP+Y\|^2_F $$ In the above, $P$ is a ...
3
votes
0answers
1k views

Is there a way to merge two trained neural networks?

Lets say I pick some network layout (recurrent and/or deep is fine if it matters I'm interested to know why), then make two neural networks A and B using that layout that are initially identical. Now ...
3
votes
0answers
116 views

Modeling products (from sales data) in a graph network for clustering and product recommendations.

I have a high level question about whether or not graph networks would be an appropriate method to model a situation I'm studying. It's been a while since I last worked on a project building/analyzing ...
3
votes
0answers
398 views

Batch-normalization back propagation equation

I have been trying to understand applying the back propagation on batch-normalization. However, the matrix multiplication and equations involved have got me confused and I feel that my understanding ...
3
votes
0answers
72 views

How can I deal with the mismatch between the vocabularies of questions and answers in a closed domain QA system?

I am building a question answering system that given a legal document attempts to answer questions related to the document. For example a tenancy agreement is given to the system and the user asks ...
3
votes
0answers
117 views

Dimensions in single layer NN gradient

Given a neural network with one hidden sigmoid layer and softmax output layer, I want to derive the gradient of the cross entropy loss with respect to the first weight matrix. This is equivalent to ...
3
votes
1answer
53 views

Machine learning on data with lots of fluctuation

I have CSV files that contain data of cache performance of a source with different workloads for a particular time period. For each time interval, data is recorded, includeing columns like ReadHits, ...