Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

how does using decision stump lead to an additive model?

In chapter 8 of ISLR it says boosting using stumps leads to an additive model. How would I derive $$f(X) = \sum^p_{j=1} f_j(X_j)$$ from $$\hat{f}(x) = \sum^B_{b=1} \lambda \hat{f}^b(x)$$?
0
votes
1answer
32 views

Why does my model consistently perform worse in cross-validation?

Okay so I run this model manually and get around 80-90% accuracy: ...
0
votes
2answers
16 views

Is softmax an activation function?

Is softmax an activation function? Because it is usually used in the output layer, why?
0
votes
0answers
14 views

Why is the step length by default equal to 1 in gradient boosting?

On ESL p.359, it explains steepest descent: But in 10.37, it is trying to minimize the distance to g_im. It looks like the default step length is 1. Why is it so?
0
votes
1answer
16 views

Quantile loss 50th is MAE, is it?

I'm not sure the above sentence is true, but I read it here, here and here that quantile loss function percentile 0.5 is MAE(mean absolute error), Is it true? and How?
-1
votes
0answers
20 views

true false questions, Help [closed]

Here are a few true false questions: Here is what I tried: There exists a data distribution and model, such that the model will generalize well to unseendata, even if the training set only contains ...
0
votes
1answer
14 views

Does the order of the columns in an CSV file affect the performance of my neural network?

It's a classification problem. I have a big dataset ( CSV file) of flights, where each flight is depicted as a set of variables ( ...
1
vote
0answers
9 views

Why does a class weight fraction improve precision compared to undersampling approach where precision drops?

I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
1
vote
0answers
24 views

Scikit learn Decision Tree not deterministic

I am doing recursive feature elimination and cross-validated selection (RFECV) in order to get the best number of features. As I will be comparing different hyper-parameters and methods in dealing ...
0
votes
0answers
20 views

Output of xgboost() while optimization is not very intuitive

I am running xgboost() on a data set with a data set with below columns. ...
0
votes
0answers
11 views

Wide feature matrix but few examples

I have a data set of 125, with only about 25 (20%) positive cases. The features, lets call them Feature1, Feature2 up to Feature250, can be easily grouped (since they all describe responses to ...
0
votes
1answer
21 views

What does State of Art Result means in context of ML/DL

Wondering what it means to have a state of art result.Is it a relative term or a standard? For Exmaple: If I have developed 2 models one with higher accuracy can i say i have achieved state of art ...
2
votes
3answers
56 views

Which image format is better for machine learning .png .jpg or other?

I'm trying to train a neural network with images. Since I'm extracting images from a video feed I can convert them either to .png or .jpg. Which format is preferred for machine learning and deep ...
0
votes
0answers
8 views

k-means with soft constraints (KSC) algorithm: how to minimize objective function?

I'm learning about the KSC algorithm as described in "Clustering with Missing Values: No Imputation Required" by Kiri Wagstaff. Here's a small dataset to use as an example: ...
0
votes
0answers
22 views

$R^2$ of Log transformed data is positive, however that of reversed transformed data is negative

I am running an XGBoost model with a continuous target variable. With ~200 features I am getting a Test $R^2$ of 0.54. By looking at the distribution of the target variable, it appears it's highly ...
0
votes
0answers
11 views

Is it wrong to use categorical_crossentropy instead of binary_crossentropy for binary classification? [duplicate]

I was trying to build a CNN model. Data: 1) Consists of time series data of minute-wise water temperature to predict if there is high level of bacteria growth(label Y) in the water or not(label N). ...
0
votes
0answers
12 views

whiteness vs Uncorrelatedness

While studying ICA in the book by Aapo Hyvarinen I found the following scentence: "A slightly stronger property than uncorrelatedness is whiteness. Whiteness of a zero-mean random vector, say y, ...
1
vote
0answers
13 views

How to use weights from our data set properly? Should we use them at all?

The census data set I'm using: https://archive.ics.uci.edu/ml/datasets/Adult So, I'm currently using this census data to make observations and predict whether someone is married or not. However, when ...
0
votes
1answer
23 views

Is CNN capable of extracting the descriptive statistics features

I was trying to build a CNN model. I used time series data of daily temperature to predict if there is risk of an event, say bacteria growth. I calculated the descriptive statistics of the time series,...
0
votes
1answer
19 views

Modeling Without Dependent Variable

I’m trying to figure out this problem where I want to calculate the probability of a set of people underpaying a service. The service needs to be paid as a percentage of people's income. The issue ...
0
votes
0answers
4 views

UP-sampled umbalanced data-set get way too much good results?

I have an imbalanced data-set. I up-sampled it then I used train_test_split to split the data-set 20% testing and 80% training (shuffle mode activated and random_state set to 42 for instance). the ...
0
votes
1answer
16 views

Is K = 1 is good for KNN, when error is min, accuracy is max and even AUROC is Max for that value of K?

I am getting highest Accuracy for K =1 in KNN, Max AUROC, and lowest Error. however, I was taught that when K = 1, then its always going to be over-fitting mode, and hence I am asking the question is ...
0
votes
0answers
7 views

What is threshold in ROC curve? [duplicate]

Whenever I read about ROC, people say that it is graphical representation of True Positive Rate value and False Positive Rate at various threshold. Whenever I read in detail, people explain that ...
1
vote
1answer
33 views

An interesting task on machine learning

There are 5 programs. Each program is a binary classifier, which classifies letters - "Spam" and "Not spam." All classifiers determine the category correctly in 60% of cases, regardless of other ...
0
votes
0answers
9 views

How are variational autoencoders and stacked variational autoencoders similar and different compared to one another?

The only source I've found on the issue is from a Bachelor's thesis, which states that they both use a similar architecture and a generative framework to learn means and standard deviations of the ...
0
votes
0answers
13 views

I have capped my response variable, should I calculate my RMSE/MAE/MAPE with the true values capped or not?

So, I have trained a model in my train set with the response variable with a superior limit. Because, the peaks are not important for my analysis. And if I dropped it, I would lost a lot of data. ...
0
votes
0answers
6 views

Classifying single individual with ML with caret [closed]

After saving and loading a trained and tested algorithm (classifier), how can I classify (ill vs healthy) an unlabeled individual on the set of predictors used for training the model? I can not find ...
0
votes
0answers
3 views

How to interpret the precision score of a model when the problem is an anomaly detection?

In a binary classification with a balanced label, the precision score is usually considered “positive” if it performs better than a random guess, i.e a precision above 0.5. When the event (label = 1) ...
1
vote
0answers
36 views

How to predict future sequence data?

I am currently doing a signal processing project. However, this is very different to projects I did before and I am struggling to find a good start point to do this. My problem is as follows. I ...
0
votes
1answer
28 views

What are the weights assigned to the features (coefficients in the primal problem) in SVR?

In case of linear kernel, how to interpret the weights in the formulations of nu-SVR? I am using nu-SVR to estimate the parameters in GARCH(1,1) model:$ \sigma_t^2 = \omega + \alpha y_{t-1}^2 + \...
0
votes
0answers
27 views

ROC curve under diagonal?

I trained an SVM to classify images based on some extracted features (using the ISIC dataset). The resulting ROC curve produced by sklearn looks like this: I have don't quite understand the line for ...
16
votes
4answers
1k views

Is leave-one-out cross validation (LOOCV) known to systematically overestimate error?

Let's assume that we want to build a regression model that needs to predict the temperature in a build. We start from a very simple model in which we assume that the temperature only depends on ...
0
votes
0answers
20 views

Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertunned ML trees etc. ...
1
vote
0answers
13 views

What are the limitations of non-negative matrix factorisation when reducing the dimensions of a data set?

From what I understand NFM (non-negative matrix factorisation) is constrained by the factor that it only supports data sets with non-negative values when reducing the dimensions of a data set. ...
0
votes
0answers
13 views

How does ReLU provide non-linearity? [duplicate]

ReLU is linear or 0. How does this property help NN to learn higher degree polynomial function? If the output is always positive then it is same as linear and I can always write output as a linear ...
2
votes
0answers
19 views

Relationship between generalization error and prior variance

Suppose that we have a Bayesian linear regression with prior on the weights $p(w)=N(0,\sigma_{w}^{2})$ Our goal is to use that Bayesian linear regression for future predictions. In order to do that ...
0
votes
1answer
26 views

Decision tree that fits a regression at leaf nodes

Is there any academic work on any Decision Tree that fits a regression at its final leaf node? For instance, suppose I have 100 features (X), and use them build a tree with 3 depths such that I have ...
0
votes
0answers
18 views

Order-insensitive variable-length sequence encoder?

While brainstorming on a new project, I stumbled upon a problem. I would like to create a neural network to encode a sequence of objects and get the gist of the sequence. However, the sequence is of ...
0
votes
0answers
25 views

What is the purpose of grids in YOLO?

Considering the YOLO algorithm. Assume: Input image is n x n x 3 Number of anchor boxes is m For each anchor box, we have 1 (pc = probability of object) + 4 (4 variables to predict the bounding ...
0
votes
0answers
16 views

Training a multi-layer perceptron (MLP) with a modified basis function

Consider a simple 3-layer MLP such as this. Each hidden layer implements y=xw+b where y is the output activation matrix of the ...
0
votes
0answers
4 views

Using supply as feature in price predictor model

On a machine learning model that outputs the optimum price of a product (ex: cars listed on some website), would it make sense to use the number of instances of that product as a feature? In the case ...
2
votes
0answers
35 views

Terminology question: distinguishing two meanings of “loss function”?

I've heard people use "loss function" to refer to 2 different things: 1) A real-valued function of a label, $y$, and a prediction $\hat{y}$. 2) A real-valued function of a parameter $\theta$; this ...
0
votes
0answers
32 views

Why using variational inference to do minimization?

I'm reading a paper on conditoinal random fields. They arrive at a formulation for energy, and they go like this: "minimizing this is intractable" What does that mean? I heard about intractable ...
0
votes
0answers
17 views

Analysing arrays of image data with Machine Learning Models

I am trying to do Machine Learning on arrays or vectors describing images. The target variable is a category I am trying to predict. I have multiple features that each contain arrays describing the ...
0
votes
0answers
12 views

Extracting metrics from natural language

Imagine a text like The revenue amounted to mEUR 124 during the year while last year it resulted in mEUR 100. I want to use ML methods to extract two outputs like ...
-1
votes
0answers
18 views

Does a pca plot gives me a relationship about my future importances?

I am searching for outliers and wanna have some graphical support. Let's assume I have a original dataset with several columns (numbers for example 7). And I do a pca decompisation. So my code would ...
1
vote
0answers
13 views

Reference: Data-Dependent Early Stopping Criterion for Deep Learning?

In the context of non-parametric regression, this paper provides an data-dependent rule for optimal early stopping, when learning an unknown function $f^{\star}$ lying in some RKHS. Here, one stops ...
-5
votes
0answers
31 views

Why NN works better SVM? [closed]

is Neural Network Ensemble give best prediction over other prediction models or algorithms? If yes, what type of Neural Network?
0
votes
0answers
16 views

Derivation of Bayes classifier in Murphy's book

I am reading Kevin Murphy's Machine Learning book (MLAPP) and want to know how he got the expression for the Bayes classifier using minimization of the posterior expected loss. He wrote that the ...
0
votes
0answers
7 views

Could someone use this concrete data set to illustrate how to compute the average Gain?

Chapter 3, Page 86 of Tom M. Mitchell. Machine Learning (free) says One practical issue that arises in using GainRatio in place of Gain to select attributes is that the denominator can be zero or ...