Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
0 votes
1 answer
42 views

What is the pseudocode for the fastest possible k-nearest-neighbors (KNN) algorithm? [closed]

I have a BERT model that's fine-tuned so that given a sentence in my X column, the model gives a vector that approximates the corresponding sentence in my multidimensional Y array. I'd like to use the ...
0 votes
1 answer
43 views

which one between XGBoost and neural networks is more interpretable? [closed]

I have heard people say, "One of the disadvantages of neural networks is that they are generally less interpretable". But I wonder, how is another model, such as XGBoost, more interpretable ...
  • 3
0 votes
0 answers
3 views

Transfer factors information from one matrix to another in the non-negative matrix factorization

I have two dataset X= [r1 x f1] and Y = [r2 x f2] Here f1 and f2 are the features such that f1>>f2 and the common features between f1 and f2 is around ~200. I am interested to know a common or ...
2 votes
0 answers
50 views

Negative Log Likelihood (NLL) reserved for a classification in PyTorch is weird?

I'm curious as to why the Negative Log Likelihood (NLL) loss is used for classification tasks in PyTorch (see here). The negative log likelihood is a much more general notion than a measurement of ...
  • 91
1 vote
0 answers
15 views

Elbow plot and plot of average silhouette width disagree with each other

I am fairly new to using clustering. On the data science course I am on, we recently covered agglomerative clustering and k means clustering. I have created a toy example to see if I can use R to ...
0 votes
0 answers
12 views

Correctly evaluating unsupervised learning model

I am trying to compare various unsupervised machine learning models to detect anomalous water consumption in each user's house. Now I have 10 datasets (minutely data, no anomalous points) that have no ...
1 vote
1 answer
48 views

In learning theory, why can't we use Hoeffding's Inequality as our final bound if the learnt hypothesis is part of $\mathcal{H}$?

This question has similar answers somewhere, but I do not understand them still: Read but do not understand Read but do not understand In the notes here, we see the definition of PAC Learning: ...
  • 715
1 vote
0 answers
13 views

If the most likely value is that which minimizes squared-error, what are the possible distributions?

Gauss uniquely characterised the 1D normal distribution by asking for a distribution that: is symmetric is decreasing on either side of some center point $\mu$ has the data likelihood maximized by ...
  • 121
0 votes
0 answers
35 views

Reinforcement learning with simple integer arithmetic actions

Motivated by an experimental investigation in pure mathematics, I am interested in the following type of reinforcement learning problems: The state is a collection of N integers, where there are no ...
0 votes
0 answers
23 views

Find correlation between continuos predictive features and a continuos target feature

In my data, I have about 10K predictive features (genes), and one target feature (age). I want to predict the ages according to the genes. The rows in the data are the patients. To do so I plan to use ...
  • 413
0 votes
0 answers
17 views

What are Dense and Sparse Models?

I'm watching an explanation of the SHAP algorithm by its author S Lundberg. At 22:45 in the video, one of the audience members asks a question about the models behind his graphs. He explains that one ...
  • 247
3 votes
0 answers
62 views

Is the transformation implied by a positive-type kernel well-defined?

I’ve been trying to get my head around the particularity of the Hilbert space that a positive-type (equiv. positive definite) kernel represents an inner product on, and was hoping for some help in ...
3 votes
1 answer
62 views

Why do the same set of data get different R2 calculated by three methods (r2_score & fit trendline in excel & linear regression in spss)?

For the same set of data x1=1, y1=3 x1=2, y1=2 x1=3, y1=1 calculated by r2_score: from sklearn.metrics import r2_score ...
1 vote
0 answers
25 views

How to train a network with multiple inputs and multiple outputs [closed]

How to train a network with multiple inputs and multiple outputs I have a dataset of (400,256) 400 examples with 256 features. and each example has its corresponding output array (400,256) 400 ...
  • 159
0 votes
0 answers
10 views

What does it mean when you scale categorical features and it increases the F1 score on validation data?

I am working on a dataset with some categorical features. Those categorical features are encoded as numbers. For example: I have a feature with the name ...
  • 1,330
1 vote
1 answer
54 views

How to know if features chosen are right?

How can we ensure that the chosen features can lead us to high accuracy if we made proper modifications in model architecture & hyper-parameters using the selected features, i.e. how can we make ...
  • 117
0 votes
0 answers
23 views

How to maximize Precision for each class in target instead of for the whole multiclass classification model using Hyperopt in Python?

I try to build multiclass classification Machine Learning model in Python. I use Hyperopt to tune my hyperparameters as below: 1. Define Parameter Space for Optimization ...
  • 141
0 votes
1 answer
14 views

How to identify deterministic functions from input-output pairs?

Suppose that we are given a sample of input-output pairs of a deterministic function. To make it concrete, I can generate input-output pairs of a function I myself created (for example $y = 240 + 20x ...
  • 1,062
0 votes
0 answers
24 views

What does this notation mean? $\mathbb{P}_{x\sim \mathcal{D}}$: Clarification in the definition of generalization error $R(h)$

I was reading the book foundations of machine learning by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar, and I came up with this definition of generalization error: I dont understand what ...
  • 101
0 votes
0 answers
8 views

How can I create a similarity network from a point cloud in vector space for classification purposes?

Are there any relevant techniques or approaches, such as k-nn graph or feature combination, that could be used to establish a similarity distance to create a graph to be used for gnn node ...
1 vote
1 answer
27 views

What are the modes of a dictionary / transform basis?

So, I'm reading Steven Brunton's book, "Data Driven Science & Engineering", and I'm trying to understand what he means by mode in this following excerpt: Most natural signals, such as ...
0 votes
0 answers
24 views

Normalisation methodology and batch effect

I have a question about the normalization of datas and batch effect ... For example, here is a dataset : ...
  • 11
1 vote
0 answers
13 views

bracey-vim cannot detect previous directory( ../ ) please help [closed]

When I try to link a file or image from a previous directory cannot load it in the browser says the error. this is my directory ...
  • 11
1 vote
0 answers
9 views

Convolutional Neural Network weight keeps growing during training

I am training a CNN. The whole training process works well. Now the loss and validation metrics are fluctuating and no longer drop, but the maximum absolute weight of convolution layers continues to ...
0 votes
0 answers
15 views

Normalization and patient effect

I have a very simple question but I don't really have answer myself... I have a set of data like this : ...
  • 11
1 vote
1 answer
76 views

What's the difference between a Permutation and a Perturbation?

I frequently come across the terms Permutation and Perturbation within the field of explainable AI. I understand that both terms refer to methods that make changes to a sample's features. However, I'm ...
  • 247
2 votes
2 answers
56 views

How to initialize final layer to get a good starting loss?

In this post Karpathy says verify loss @ init. Verify that your loss starts at the correct loss value. E.g. if you initialize your final layer correctly you should measure ...
  • 274
0 votes
0 answers
9 views

How to get the dynamic threshold to detect rare categories from the univariate categorical variable?

I was working on combining the "rare" categories into a single category/group "others" automatically in a univariate categorical variable. Suppose I have A, B, C, D, and E ...
5 votes
1 answer
58 views

Deriving Log Marginal Likelihood for Gaussian Process

I am trying to evaluate the following integral marginalized across all possible functions. $$\mathbb{P}(y|X,\theta) = \int \mathbb{P}(y|f)\ \mathbb{P}(f|X,\theta) \ df$$ In G.P. we assume prior to be ...
1 vote
0 answers
33 views

Deriving vectorized back propagation

I'm trying to derive vectorized backpropagation from mostly first principles, but I'm having trouble marrying how this paper explains backpropagation with the derivative of a loss function with ...
1 vote
0 answers
39 views

What is the best approach for feature selection if I have 65536 features

Don't want to write a big paragraph and for the beginning, what I would like to say is I started doing a machine vision project and I flattened my 256x256 photos into a vectors of 65,536 features. ...
  • 111
0 votes
1 answer
18 views

Can this approach be used for machine learning using train-test split?

So let's say I have a dataset with 1000 samples, 20 cols. Regression problem. I use train-test split, say 80-20% I create a Model, lets say Random Forest. I use gridsearchCV to find the best model ...
0 votes
1 answer
31 views

Using cross-sectional data for OLS/logit models

I have a cross-sectional dataset with the data example below, where the variable (id) refers to each individual in the df and rows represent the different number of Reddit posts written by each ...
0 votes
0 answers
8 views

How to evaluate a dataset in terms of discriminative features across classes?

Is there a way in machine learning to evaluate datasets in terms of how difficult/easy they are to handle for classification purposes? In other words, to calculate a number that gives an indication of ...
0 votes
0 answers
37 views

Trouble with understanding alpha and beta in HMM

I'm implementing HMM myself and I'm stuck with this concept. Let T be the total time steps. $\pi$ be the initial probabilities. A be the transition matrix. B be emission matrix. $\alpha_{t,i}$ ...
  • 13
1 vote
0 answers
10 views

AdaGrad: motivation and complex values

I have started learning the math behind the AdaGrad optimizer, and two questions emerged that I cannot find any answers on the Internet: It is said that in AdaGrad "parameters associated with ...
0 votes
0 answers
12 views

Correlation Analysis among groups

I am trying to do a Bivariate correlation analysis using stats This is how my sample data looks like Performance Rating Activity Count Top Performer 54 Top Performer 66 Top Performer 74 Top ...
2 votes
0 answers
46 views

Confused about The Realizability Assumption

From the book "Understanding Machine Learning: From Theory to Algorithms", The Realizability Assumption is defined as follows: There exists $h^{\star}\in \mathcal{H}$ s.t. $L_{(D,f)}(h^{\...
  • 21
0 votes
0 answers
23 views

Low F1 score in a propensity model

I'm trying to model a propensity problem to a product, which is highly unbalanced (99.9% against 0.1%). Here is the details of my dataset: The target is a flag which indicates if the client bought ...
  • 1
2 votes
1 answer
84 views

Why is the cross entropy of the same probability distribution not 0?

From what I've been reading, if there is no underlying difference between the 2 probabilities distributions we would have perfect entropy. I'm putting an example below. Can anybody explain why the ...
1 vote
1 answer
46 views

What do I make of all classification scores being equal to 1?

I've built an XGBoost classifier with following code: ...
  • 111
0 votes
0 answers
15 views

Parametrize the variance of the gaussian posterior distribution in vae

I noticed that in most of the implementations of a variational autoencoder with gaussian posterior, the variance of the gaussian is not learned during training. The decoder usually outputs only the ...
  • 1
2 votes
1 answer
89 views

How do we find weight and b value of SVM classifier for XNOR operation by hand?

Problem statement Show that the Boolean function (x1 ∧ x2) ∨ (¬x1 ∧ ¬x2) is not linearly separable (i.e. there is no linear classifier sign(w1 x1 + w2 x2 + b) that classifies all 4 possible input ...
  • 71
0 votes
0 answers
36 views

Using simulated data to test ML algorithm

For a project I’m working on looking at dichotomous outcomes, we are comparing the ability of different ML algorithms to detect specific culprit factors that are associated with an outcome of interest ...
  • 127
4 votes
3 answers
1k views

How can q(z,l,u,c|x) be factorized to q(z|x)q(l|x)q(c|z)q(u|c,z)?

How can q(z,l,u,c|x) be factorized to q(z|x)q(l|x)q(c|z)q(u|c,z)? I don't know what assumptions are behind this equation, which is I want to know. I think normal chain rule of probability is in the ...
1 vote
0 answers
29 views

How useful is PCA on its own? [closed]

In my machine learning course we have covered the key ideas behind principal component analysis. To round this part of the course off, we have learned to interpret the results of PCA, specifically ...
0 votes
0 answers
42 views

How can we deal with features that are vectors in machine learning?

It is often the case that the predictors are variables that take continuous or categorical values in machine learning. However, if the predictors or some of them are vectors, how can we deal with it? ...
1 vote
1 answer
42 views

What is fixed and what varies in the bias-variance decomposition?

I am reading about the bias-variance decomposition from An Introduction to Statistical Learning with Applications in R (Second edition at page 34). It states that $$Y = f(X) + \epsilon$$ where the ...
  • 221
1 vote
0 answers
24 views

Interpreting results of PCA [duplicate]

I've recently started an introductory machine learning course and the first topic we have covered is prinicpal component analysis (PCA) and overall I am finding the whole topic quite tricky to wrap my ...
0 votes
0 answers
13 views

Why is a separable SVM solution optimal for separable data?

Let's consider the separable problem in Support Vector Machine fitting as a special case of the non-separable problem. My terminology will be consistent with Mohri et al (2018), so I use 'risk' for '...