Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

68
votes
2answers
15k views

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
2
votes
1answer
191 views

Bias-variance docomposition of linear model fit in 'The Elements of Statistical Learning'

In section 7.3 of 'The Elements of Statistical Learning', the authors have shown the expression for bias-variance decomposition of linear model fit: However, I get a slightly different expression for ...
2
votes
1answer
26 views

Why is random sampling a non-differentiable operation?

This answer states that we cannot back-propagate through a random node. So, in the case of VAEs, you have the reparametrisation trick, which shifts the source of randomness to another variable ...
0
votes
0answers
12 views

Gaussian process - Why adding data points cannot increase the predictive bias?

I've seen this question here: How to increase variance in Gaussian Process regression? But a proof isn't provided. I'm looking at this book: Rasmussen Williams 2006 Gaussian processes for machine ...
1
vote
1answer
11 views

Analysis on cluster change

I have 50 datasets each for every year from 1961 to 2010. These datasets keep data about GDP, mortality, natality, etc. My intent is to apply clustering for each dataset and then compare clusters. ...
1
vote
0answers
33 views

Why using RMSE as loss function in logistic regression takes non convex form but doesn't in linear regression? [duplicate]

I am taking this deep learning course from Andrew NG. In the 3rd lecture of 2nd week of the first course, he mentions that we can use RMSE for logistic regression as well but it will take a nonconvex ...
5
votes
1answer
548 views

Is it possible to use LASSO regression with multi-levlel data?

I have real-time monitoring data where participants report on a variety of variables four times per day for a month. Is it possible to use LASSO regression (e.g,. glmnet r package) with this data? I'm ...
1
vote
1answer
217 views

Out-of bag error in Random Forest

I am trying to code my own, simple version of RandomForest function in R for learning purposes. However I have a hard time understanding the concept of the out-of-bag error. Is it simply done by ...
0
votes
1answer
367 views

Difference Between Adaline and Delta Rules

What's The difference between the Adaline and Delta rules in Neural Networks ?
0
votes
0answers
20 views

Statistically evaluating classification accuracy of machine learning model

Let's say I'm trying to evaluate a classification algorithm and suppose there are $m$ data points in my test set. Here's my understanding so far: assuming my evaluation metric is the classification ...
4
votes
1answer
34 views

Classifier which minimizes inaccuracy

I recently interviewed for a machine learning job which involved very mathematically rigorous questions. This is one of them, which I'm still very confused about. Question: Given a data generating ...
0
votes
0answers
13 views

what is the difference between a multilayered autoencoder and a hierarchical latent variable model?

I have been trying to understand how hierarchical latent variable models are different from multilayered autoencoders and in specific the argument below Autoencoder networks resemble in many ways ...
2
votes
1answer
49 views

Effect of class imbalance on logistic regression (mathematical basis)

A number of posts, and papers, state that logistic regression (LR) is robust in the face of class imbalance. Unless the imbalance is extreme (e.g., events=0.01 or less), with adequate sample sizes ...
0
votes
0answers
32 views

Is it possible that PCA works better without data scaling? [duplicate]

I am a beginner. I have a dataset of 1700 samples with 4 features and I have to perform Hierarchical Clustering (the agglomerative version) and I need to decide whether or not to scale the data and ...
1
vote
0answers
85 views

Transfer Learning for Multivariate Regression

As I understand it, transfer learning is termed as using the parameters of a pre-trained model, which was initially trained on a particular 'source' task, and have it train on another related 'target' ...
0
votes
0answers
3 views
0
votes
0answers
19 views

How to compare Accuracy of two RandomForest models? (Chi Sqr or Cohen's H?)

I've got two dataset which have exactly the same structure (15 features, 1 class variable with 7 categories) and roughly the same amount of observations). I trained a Random Forest with the full ...
2
votes
2answers
349 views

scoring metric for regression that does not weight outliers heavily

I'm using the root mean squared error (RMSE) as a metric for tuning the parameters of my model in a regression problem through cross-validation. However, I'm not so much interested that all ...
1
vote
0answers
28 views

K-Fold Cross Validation: Bias in cross-validated effect size measures?

As pointed out by various authors (e.g., Hastie, 2011), K-fold cross-validation has an upward bias of prediction error. I wonder whether the same holds for cross-validated effect size measures such as ...
0
votes
0answers
9 views

SVM in the classification layer of a Feedforward neural network

I want to use SVM in the classification layer of a 2 layer feedforward neural network. Need guidance from the community on how to approach this problem. This involves capturing the features from the ...
-1
votes
1answer
24 views

Unrelated features with common target issue [on hold]

The problem is we have many unrelated dimensions with common target value. .... ..... We want to build a formula that predicts this target variable (continuous numeric): Target = Big_Model(Model1(...
0
votes
1answer
11 views

How to interpret a high sensitivity and low specificity using svm classifier?

I want to have your opinion on the performance of my SVM classifier (k-fold cross validated): Classification of class1(n=45)/class2(n=86) Accuracy: 65.4% Sensitivity: 88.2% Specificity: 22.2% AUC:...
0
votes
0answers
6 views

Scoring the difference between a family of distributions and a test distribution

Let's suppose we have a random model that I can sample to generate distributions of a certain 1D variable. I want to score the distance of a test distribution to the model in question. The distance ...
3
votes
3answers
4k views

Function with multiple local minima

I am trying to understand gradient descent algorithm by plotting the error vs value of parameters in the function. What would be an example of a simple function of the form y = f(x) with just just one ...
0
votes
1answer
16 views

Machine Learning problem - label over multiple lines

Currently I'm trying to work out a project where I would like to recognize movements from videos using machine learning and python. What I've done so far is extracting the x and y values of body ...
2
votes
1answer
49 views
0
votes
0answers
18 views

Looking for ways to transform time-series data recorded from object movement into equation describing the movement direction of the object

Looking for some time-series data transformation advice! I want to know what's the best way to transform data of 9-tuples time series data of IMU (Inertia Measurement Unit) sensor, recorded from a ...
-1
votes
0answers
11 views

How to add regularization term lambda? [on hold]

I have this equation in online-sequential extreme learning machine. how can include a multiplication regularization term lambda to govern how rapidly the network weights update please any help. ...
2
votes
1answer
2k views

SMOTE using unbalanced package in R fails on simple simulated data

SMOTE is a popular method to generate synthetic examples of the minority class in an unbalanced-class data set. I am trying out SMOTE in the "unbalanced" package in R. I am generating a simple ...
0
votes
0answers
9 views

Implement the probability distribution function of a dataset and calculate the Mahalanobis distance of any given new instance [duplicate]

I'm looking for some practical guidance, as I have a limited time to go from theory to implementation. I would appreciate concrete recommendation on libraries, tutorials or similar to implemente a ...
0
votes
0answers
6 views

Method to identify recurring patterns (motifs) in a time series after giving a reference pattern

I would like to know if there exists an algorithm using which I would be able to extract repeating patterns from a time series dataset, provided I give it a reference shape. I have included an image ...
0
votes
0answers
11 views

synthesized data to train classifier

Our dataset is relatively small (303 x 14) and so we decided to use synthpop package in R. The basic idea of synthetic data is to replace some or all of the observed values by sampling from ...
0
votes
0answers
11 views

Random effects model vs Region by region model - which is more accurate for translating to business action?

I have 100 regions and several features like [quality_rating, the density of stores in a region, the number of people who purchase, age, gender, income, lifestyle], etc. for each region. Let's say my ...
3
votes
1answer
68 views

Features that are important according to random forest are not significant when logit model was used. How to interpret?

I have a feature set for each customer [age, gender, income, lifestyle, & so on...] and a response variable say: has_repurchased. I use a logit model summary which shows income & gender to ...
0
votes
0answers
8 views

What is the scale of pseudo R squared obtained from my Logit model?

I am getting a pseudo R squared in the range of 0.01 - 0.05 when I experiment with various combination of features. I am aware of this post: McFadden's Pseudo-R2 Interpretation says 0.2-0.4 ...
0
votes
0answers
26 views

How to derive a ranking function from observations

I have a employee dataset with the following 5 details. projects completed customer ratings number of bugs reported customer complaints profit I want to rank ...
1
vote
1answer
17 views

Including the outcome variable in multiple imputation

I'm trying to perform binary classification on a dataset with missing values. I used sklearn's iterative imputer to impute these values and I got pretty good results. However, I realized that I was ...
0
votes
0answers
25 views

Can bagging outperform a random forest?

I want to predict wages and constructed bagged regression tree's and a random forest of regression Tree's. The bagged regression tree's outperform the random forest. Is this result even possible , I ...
0
votes
0answers
14 views

Machine learning for discrete probability distributions when events can be rare

I am following a paper which suggests using a restricted Boltzmann machine for learning a discrete probability distribution. I have encountered a problem, however, when the scientifically "interesting"...
0
votes
0answers
7 views

Oversampling in Uplift Modelling

I hope any of you can help me in the following matter: I am about to write my master thesis addressing the question how response and uplift modelling differ in terms of performance but also the ...
5
votes
0answers
37 views

Asymmetric or unequal misclassification costs in random forest

I have a general question about asymmetric costs. In machine learning problems, there are times when the cost of a false positive is different from the cost of a false negative. Accordingly, models ...
0
votes
1answer
345 views

On the convergence of Iterated Conditional Modes (ICM) for MAP inference

ICM is very fast but I could not find any references that contain a detailed analysis on its convergence (e.g. rate of convergence). Any suggestions please? Thanks a lot for your help!
1
vote
0answers
22 views

Proof for asymptotic error in logistic regression

Ng, A.Y., and Jordan, M.I. (2001). On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes. Advances in Neural Information Processing Systems, 14, pp. 841-8, ...
0
votes
0answers
20 views

Using sigmoid in binary DNN output layer instead of softmax?

For a binary DNN, the output is $y_0 + y_1 = 1$ since they are the probability distribution, hence the sum must equate to 1. However, I've been told that $y_1$ is sufficient to represent the output of ...
0
votes
1answer
12 views

How to interpret precision and recall for multiclass prediction?

I have a few models doing prediction with 4 classes, with the output precision and recall varying with different labels. For example I have (with the class labels being 0, 1, 2, 3 on the x axis): I ...
0
votes
0answers
16 views

What are other nonlinear transformation methods in machine learning except Neural Network activation functions?

One advantage of the MLP neural networks is the nonlinear transformation used on raw features. The popular ones used are the activation functions like Sigmoid, Tanh, ReLU, Leaky ReLU, etc. They are of ...
0
votes
0answers
9 views

How to calculate accuracy of model without having actual labels [on hold]

i have a dataset having traing.csv and testing.csv files. i have trained the model on traing data and then predict the labels for testing data. as the labels are only given in training data. there are ...
0
votes
0answers
15 views

Unsupervised Learning: Distinguishing 10 spoken numbers in Arabic

I want to develop an unsupervised learning method to identify spoken numbers in Arabic. My dataset consists of MFCC arrays. Every row consists of an array of shape(41,13), The row consists of float ...
0
votes
0answers
15 views

On Structures in Dataset [on hold]

I am working on a dataset challenge and am being asked to detect structures in the dataset. What are some ways we can define structures within the data? Is that pretty much any patterns found? Maybe ...
0
votes
1answer
182 views

How do I optimize decision tree regression algorithm implemented in R?

I'm only getting an accuracy of 59% using the following implementation calculated using the diag(sum(cm)) and sum(cm) functions. ...