Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
-1
votes
1answer
44 views

is time series analysis suitable for my dataset

I am monitoring user behavior while the user interacts with a form on a website. That form has multiple textfields from top to bottom and at the bottom it has two buttons: "cancel" and "save". My ...
0
votes
1answer
62 views

Why is the correlation of a variable and itself a histogram?

This post is visualizing the Wine dataset. You may have noticed that the figures along the diagonal look different. They are histograms of values of individual variables. We can see that the "Ash" ...
0
votes
0answers
21 views

Linear Regression in Python using gradient descent

(cross-posted from data-science StackExchange)(someone recommended that this community is more appropriate for my problem) I am trying to implement a simple multivariate linear regression model ...
3
votes
1answer
35 views

How interaction terms are treated in Gradient Boosting? [on hold]

In GAMs interaction terms have to be expressly specified as covariates, even for simple linear relationships. On the contrary, with Gradient boosting this is not nesessary because the algorithm itself ...
70
votes
6answers
34k views

Why use gradient descent for linear regression, when a closed-form math solution is available?

I am taking the Machine Learning courses online and learnt about Gradient Descent for calculating the optimal values in the hypothesis. h(x) = B0 + B1X why we ...
0
votes
1answer
11 views

What does it mean when someone says they have sparse labels?

I'm going through the Hands-On Machine Learning book with Scikit-Learn and TF by Aurelion Geron and I've come across the notion of choosing a specific loss function due to the data having sparse ...
0
votes
1answer
42 views

Could someone give an concrete example to illustrate how “empirical distribution histogram” relates to “histogram”?

This question is derived from this one, which is related to empirical distribution. I did a little bit search and then got this and this, unfortunately, none of them mentions "histogram". I've ...
1
vote
1answer
23 views

How to optimize performance of cluster model without any ground truth? [duplicate]

I had a general question on what to do when no ground truth data is available and clustering is initiated. Are there still metrics which can indicate how good or bad the clustering worked on the "...
1
vote
0answers
23 views

Is the binarization of a multi-class classification worse than a binary classification?

BACKGROUND Consider $N$ classes $\{C_1, \cdots, C_N\}$ such that the contingency matrix $\mathcal{M}^{\scriptsize(N)}$ produced by some mutli-class classifier $M^{\scriptsize(N)}$ on a test set is $N$...
0
votes
0answers
11 views

In what step should I try to find a best thread cut-off point for binary classification?

I am working on an imbalanced binary classification and wondering in what step I should find the best optimal threshold cut-off point. When I tried classifying the dataset with the normal probability ...
0
votes
0answers
14 views

How to plot the Kernel function on **one** data point with python? [on hold]

This question is derived from this one, which stuck on [Kernel Density Estimation]. I am trying to get an intuition about Kernel Density Estimation by plotting the Kernel function on one data point ...
0
votes
1answer
20 views

time series forecasting with machine learning

I have collected some data which basically encapsulates some internet traffic behaviour like average packet delivery time between two sensors, queue lengths etc. at different times of the day and ...
0
votes
2answers
389 views

How does one know if normalizing is improves reconstructions in the task of auto-encoding?

I wanted to understand the performance on an algorithm in the auto-encoding task and compare understand if normalizing the data was a good idea or not and compare the performance when the data is ...
0
votes
0answers
21 views

What an input space exactly is in the context of machine learning?

I've been confused about various "space"s in machine learning for a long time. I've checked out this, this and this post. I am trying to get understanding through some concrete examples like this ...
0
votes
1answer
23 views

How does knn regression .predict() work?

For a typical regression algorithm like linear regression, the model is y=2x+1 for instance. We can make predictions y = 3 when x=1 Picture above is an example from github.The green ...
0
votes
0answers
18 views

Maximum Likelihood Estimation and expected value

In the Deep Learning book, when Goodfellow is trying to derive the MLE equation, he scales the following equation by $1/m$: and then derives the following: How does dividing $1/m$ is turned into the ...
0
votes
0answers
6 views

Show that if $H$ is PAC learnable (in the standard one-oracle model), then $H$ is also learnable in the two-oracle model

Consider a variant of the PAC model in which there are two example oracles: one that generates positive examples and one that generates negative examples, both according to the underlying distribution ...
1
vote
1answer
265 views

Random forest permutation test: Is permutation of the training set appropriate?

I have a rather high-dimensional data set (p > 1000) with several variables ranking significantly higher than the rest in terms of variable importance (measured by Gini impurity). However, these ...
0
votes
1answer
10 views

Default threshold in cross-validation metrics - h2o R package

I created an cartesian grid of GBMs using h2o package in R and saved cross-validation metrics for each model in a data frame. So, for each model, I stored the ...
1
vote
2answers
139 views

Machine learning - PCA and KNN on rgb images are too slow

I work with python and images of tables (taken from above). My aim is to take a photo of a random table and then find the most similar tables to it in my database. Obviously, the main feature which ...
1
vote
2answers
37 views

Should you take a sample when doing EDA?

Suppose i have a large dataset, such that python graphing libraries are unable to handle. Is it a good idea to take a random sample? Specifically if it's a classification task, and where the target is ...
0
votes
0answers
14 views

How to predict the time of next trajectory for users

I am working with the Geolife dataset and would like to create a statistical model to predict when users will go on their next trip. The implication is to see if we can build a model to be able to ...
2
votes
0answers
33 views

Linear model with and without intercept [duplicate]

This question is based on Everitt et al. (A Handbook of Statistical Analyses Using R) and I am trying to answer these questions: Load the Default dataset from <...
0
votes
2answers
36 views

Is it possible to define an optimal fit?

Let assume that we have n pairs of real numbers: (x_1, y_1), (x_2, y_2), ..., (x_n, y_n) Let as also assume that ...
0
votes
0answers
30 views

Gaussian Kernel and Feature Space

I have been reading this paper for a few days. There is one section (Section 3.3) that confuses me. We start by gathering local features from training images of a particular class into a single ...
0
votes
0answers
22 views

Does probability calibration always enhance the logloss?

I'm trying to calibrate some probabilities returned by different classifiers. I have plotted the auc and logloss of each one and ...
0
votes
0answers
7 views

Pre-hoc methods to determine sample size based on known number of features and classes

What are some pre-hoc methods that allow me to determine (or roughly estimate) sample size based on the known number of features and classes? I have found this article: Optimal number of features as ...
10
votes
3answers
4k views

What algorithms require one-hot encoding?

I'm never sure when to use one-hot encoding for non-ordered categorical variables and when not to. I use it whenever the algorithm uses a distance metric to compute similarity. Can anyone give a ...
1
vote
1answer
94 views

Choosing sample size for building predictive models and machine learning

Is there a way of estimating in advance the number of observations one needs to achieve a maximum prediction error given a machine learning problem? There are a huge number of questions on CV and ...
0
votes
1answer
18 views

Can someone give a concrete example of exploitation in the context of Exploratory Data Analysis?

This post says Exploratory Data Analysis (EDA) consists of 2 steps exploration and exploitation. I know a little about exploration which uses some techniques such as data visualization to understand ...
0
votes
1answer
211 views

Dimensionality Reduction of Self Organising Maps

I've probably read any article on dimensionality reduction of Self Organising Maps but just couldn't fully comprehend this process. My understanding so far is: SOM are two-layer networks, ...
0
votes
0answers
8 views

what does negative coherence score means?

I am working on topic modeling and when I tried to see the coherence of the topics all are negative. Does anyone know what could be the interpretation behind negative score? This is the example of ...
2
votes
1answer
459 views

Why does Batch Normalization need moving averages besides to track model accuracy?

I was reading the new layer normalization (LN) paper and it mentioned that batch normalization (BN) batch normalization required moving averages. I was re-reading the paper but and it says the moving ...
2
votes
1answer
20 views

Is there a formal relation between weight regularization and compression?

In my understanding, compression, strictly speaking, means that we diminish the amount of data required to describe something, such as a model. E.g. compressing an image file means to create a file ...
1
vote
0answers
109 views

Invertibility of Random Fourier Features

Is it possible to approximately reconstruct a point $ \mathbf{x} $ in a vector space (say $\mathbb{R}^n $) given it's randomized feature map $ z(\cdot) $ and respective projection $ z(\mathbf{x})$ (in ...
1
vote
1answer
27 views

Random fourier features and Bochner's Theorem

The paper, Random Fourier Features for Large-Scale Kernel Machines by Ali Rahimi and Ben Recht , makes use of Bochner's theorem which says that the Fourier transform $p(w) $ of shift-invariant ...
0
votes
2answers
45 views

For what values of $\beta \in \mathbb{R}$ is $t(x-x')=-||x-x'||^\beta$ a kernel?

For what values of $\beta \in \mathbb{R}$ is $t(x-x')=-||x-x'||^\beta$ a kernel? I know that kernels of type $t(x-x')$ where $t$ is function that inverts the dissimilarity $x-x'$ into a similarity ...
0
votes
0answers
11 views

How to implement adamW? [on hold]

I have implemented AdamW but I am not getting good results, is there some mistake in my implementation? template void AdamW(vector &dW1, vector &dW2, vector &dW3, vector &W1, vector &...
0
votes
1answer
136 views

With an LSTM, with training samples on 0->250, should it be able to extrapolate to unseen data(e.g. 250->500)?

I'm currently training on a simple dataset: Training: [0,1,2,3,4,5...250] Test: [251-500] My training cost and expected output decreases and seems correct. However, when I test the model, my network ...
2
votes
1answer
31 views

“Learning curve” behavior comparison

I am comparing two learning algorithms using the log learning rate. I plot training data size vs. Mean Absolute Error (MAE). In the figure below, you can see method 1 shown with black line and method ...
0
votes
0answers
12 views

Bayesian inference out of partial information - Dirichlet example

Suppose we have two coins $X_1$ and $X_2$. They are possibly biased and correlated coins. The heads probability of each coins is denoted by $p_1$ and $p_2$ which we don't know at the beginning. The ...
0
votes
0answers
24 views

Isolation Forest Numerical Example

I'm looking for a proper numerical example to understand Isolation Forests Algorithm correctly. I've read the paper : https://github.com/mgckind/iso_forest/blob/master/icdm08b.pdf, but I want to ...
1
vote
0answers
19 views

How to make a custom activation function in keras with a learnable parameter?

The answer to this question is generally to implement it as a new layer and do layer = Dense(num_neurones)(previous_layer) out = TheActivationFunction()(layer) ...
7
votes
1answer
5k views

Can a GAN be used for data augmentation?

Can a generative adversarial network (GAN) be used for data augmentation (i.e. to generate synthetic examples that are added to a dataset)? Would it have any impact on the performance of a model ...
2
votes
1answer
385 views

Expected value of error in neural network

I wanted to take a look at the properties of the error vector that is propagating during backpropagation. The error vector $\boldsymbol{\delta}$ at layer $i$ is nothing more than the derivative of ...
3
votes
1answer
142 views

How can one implement PCA using gradient descent?

I have to implement PCA using gradient descent and stop at convergence. I am not able to find the objective function. I know that the aim of PCA is to reduce the $n$-dimensional matrix to $k$ ...
0
votes
1answer
11 views

Interpret the clustering results of Weka to measure the performance [on hold]

I'm having the Boston dataset, where it's class variable in the housing price. So I think regression is more suitable for this dataset, so we can predictions. I'm using Weka for this. I used several ...
3
votes
2answers
243 views

Are loss functions what define the identity of each supervised machine learning algorithm?

For supervised machine learning algorithms (ie: regularized logistic regression, SVM, decision trees, etc), are their specific loss functions the main/only reason they differ from one another?
2
votes
2answers
139 views

improving classification accuracy of the dataset as a whole by considering classifier distributions

Overview I'm new to machine learning so apologies if I misuse terms. I have an idea to improve my classification analysis that I feel is not terribly unique, but I can not find a reference to such a ...
1
vote
0answers
46 views

Distribution to model Binomial distribution with parameter $p$ in trial $n$ dependent on result from trial $n-1$?

I'm wondering how one can model a Binomial distribution as described in question. E.g., $p = 0.5$ for trial $n = 0$; $p(n+1) = p(n) + 0.01$ if for trial $n$ Bernoulli(p(n)) samples to 1, else $p(n+1) ...