Questions tagged [machine-learning]
Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.
19,417
questions
0
votes
0
answers
11
views
multiple time series dimensionality reduction
I have a time series for each of a couple hundred patients, around 10-20 samples per patient, unevenly distributed through time, with over 40000 columns per sample. The target feature is the level of ...
1
vote
1
answer
42
views
I need to get 100% accuracy on my training data
I know this is undesirable in most cases, but I have a very niche case where I must achieve 100% accuracy on training data. I do not care about unseen points, I only care about given X points what is ...
- 11
0
votes
0
answers
10
views
Does it matter which variable I assign 1 or -1 in a perceptron machine learning algorithm
I am using perceptron machine learning to solve the binary classification problem A vs B. For this I have to assign the actual values of A and B to either 1 or -1 to be able to use perceptron. Does it ...
1
vote
0
answers
20
views
Time-series prediction using unevenly spaced data
Suppose I have unevenly-spaced minute-precision data in a specific window (11am-3pm) per day for many days, that looks like below:
file1:
timestamp
value
3/20/2023 11:00
183
3/20/2023 11:01
190
3/...
- 11
0
votes
0
answers
5
views
Why Out-of-bag score used for Bagging Ensembles and only for Bagging?
I study ensemble machine learning and I have noticed OOB (out-of-bag) score in some implementations. I understand the concept but I have some questions about the general usage:
Why is it used for ...
- 725
0
votes
0
answers
7
views
Silhouette Score for ordered clusters
My clusters are arranged according to a time series, and I want to compute the silhouette score for the clustering performed, considering that they follow an order. Therefore the nearest cluster to ...
- 101
0
votes
1
answer
20
views
Why do the error derivatives become small if we start with a large learning rate?
In these slides from Hinton (https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf) there is this statement:
I don't understand why "The error derivatives for the hidden units ...
- 1
0
votes
0
answers
24
views
Is the multiple correlation coefficient (R) undefined in the case of negative determination coefficients (R²) - Neural network models?
I noticed that in some low performance models of neural networks, the value of R² (coefficient of determination) can be negative. That is, the model is so bad that the mean of the data is better than ...
- 1
0
votes
0
answers
8
views
Multivariate Timeseries Forecasting - Add external data from the future
I would like to add external variables to timeseries forecasting. This variable partly refers to the future - so it is withing the timerange, where the target variable should be forecasted. Based on ...
- 1
0
votes
0
answers
32
views
Is cross-entropy a reliable measure of similarity between probability distributions? [closed]
I'm working on a project that involves measuring the similarity between distributions using cross-entropy. However, I'm a bit confused about the reliability of this measure.
For example, let's say I ...
- 1
1
vote
1
answer
20
views
Calculating KL divergence with entropy and cross entropy for VAEs
When looking at implementations of VAE's online, specifically the KL divergence loss, the formula used is:
$$ KL\hspace{1mm} Loss = -\frac{1}{2}(1+\log{\sigma^2}-\mu^2-\sigma^2) $$
or some variation ...
- 13
0
votes
0
answers
23
views
What mean and variance should we use for inferring results using linear regression? [duplicate]
While training a linear regression model, it is advised to standardize the input features using the mean and std deviation of the train input features.
...
- 1
1
vote
0
answers
14
views
Which error metric should I use for summed estimates?
I built an ensemble regression model (Random Forest + KNN + SVM) to predict biomass based on environmental conditions (biomass is strictly positive but continuous).
I now would like to use this model ...
- 11
0
votes
0
answers
6
views
Question model selection & assessment in Nested CV
If I got it correctly, in nested cv we have 2 layers, k1 and k2.
In K2, we choose the best model, with its hyperparams in K1 we do model assessment.
How fair is it to estimate the generalization error ...
- 1
0
votes
0
answers
8
views
What kind of predictive model can I use to recommend one intervention vs another?
Study question: build a model that predicts if a patient would benefit from treatment A vs treatment B.
Outcome: numerical survey score at 24 months
Other variables: demographics, potentially imaging.
...
- 1
1
vote
0
answers
15
views
How do I fit an equation to this butterfly curve?
I have data which I plotted to visualize. I cant seem to fit any sort of sensible curve to this plot and I wondered how do I approach this task?
- 11
2
votes
3
answers
128
views
What are the methods to increase the dimension of a feature space?
Is there a way to increase the number of dimensions through feature transformation in machine learning? If so what are the techniques involved?
0
votes
0
answers
17
views
Non-stationary Random Fourier Features
Random Fourier Features (RFFs) were introduced by A. Rahimi and B. Recht in their 2007 publication Random Features for Large-Scale Kernel Machines. RFFs are based on Bochner's theorem, which applies ...
- 1
12
votes
8
answers
2k
views
Practical usefulness of PCA
I asked a similar question in the past, but I've thought about the message I am trying to convey a bit more and feel I can articulate it better. For context, I am on an introductory course in machine ...
1
vote
1
answer
17
views
Does feature selection and model testing have to be coupled in each fold of the cross-validation?
Quick overview of my data and aims:
I have two groups, 50 samples per group, and 6000 features. I want to find the minimal amount of features capable of distinguishing both groups. I know the sample ...
- 11
1
vote
2
answers
35
views
Can I use K-Means to group customers based on a single variable?
I have a test dataset of 11m records. The dataset contains a global customer id and spend figure.
I need to group customers into the following categories:
0 Low
1 Low/Med
2 Med
3 Med/High
4 High
I ...
- 11
3
votes
0
answers
13
views
Fraud detection model retraining
I’ve got an interesting question at one interview.
Assume that we already trained and deployed some fraud detection model for some online service, and it has helped us to decrease amount of fraudulent ...
- 31
0
votes
0
answers
27
views
My loss has a non-differentiable point
I had to design a loss function max(0,x). It's not differentiable at x=0. In order to train it with gradient descent, what should I do?
I have learned that subgradient can be used instead, so does it ...
1
vote
1
answer
64
views
Is the iid assumption in Linear Regression necessary?
In linear or logistic regression, we have the following (adapted from Foundations of machine learning.):
As in all supervised learning problems, the learner $\mathcal{A}$ receives a labeled sample ...
- 715
0
votes
0
answers
13
views
what type of machine learning or neural net algorithm would i use for predictions about the shape of a plot
if a data set is set up such that it has 5 independent variables and an array of 201 elements of "smooth continuous" data as the dependent variable. I was wondering if there was a model ...
3
votes
1
answer
79
views
What are some good calculus resources relevant for Machine learning researcher aspirant?
I am trying to self-taught myself on Calculus for machine learning and read the book by Spivak. But it is too rigorous and need a lot of time to finish it.
As far as I am concerned, Calculus is only ...
8
votes
2
answers
393
views
What is the algebra showing the logistic and log loss to be equivalent?
This question discusses two equivalent ways to express the canonical loss function for a logistic regression, depending on if you code the categories as $\{0,1\}$ or $\{-1,+1\}$. In the following, let ...
- 46.4k
1
vote
0
answers
62
views
Can XGBoost handle a custom objective where the 2nd derivative can be negative?
I am going through the introduction to XGBoost page, and there is a section where they derive the optimal value of the leaf node, for a given tree structure.
To quote the specific section,
In this ...
- 3,304
0
votes
0
answers
16
views
control=list(maxit=20) this argument in optim function does not work [closed]
I use this in my optim function like this
optimum_theta <- optim(par = theta_initial,fn = cost,method = "BFGS",control=list(maxit=20))
During the ...
- 1
1
vote
0
answers
22
views
The multiple testing problem
I am learning about the multiple testing problem and I'm confused on the number of expected false positive.
So let say I set the p-value to be 0.05, and I run 1 test with 100 samples, then i would ...
- 11
7
votes
2
answers
106
views
How can a statistician be relevant today? [closed]
Apologies if the question is unsuitable for this site. Please direct me to the appropriate place, and I will take this down.
I am a statistician, and I have been struggling to find a meaning to the ...
2
votes
0
answers
32
views
Package for Multidimensional Density Estimation
I may be missing something obvious, but is there a python package that can reliably do density estimation of a PDF in high dimensions (e.g. 512)? I know of scipy's ...
- 21
2
votes
0
answers
12
views
In practice, should I use k-fold cross-validation or repeated random sub-sampling validation as my default choice of evaluating the model performance?
I was wondering if someone can shed some light on which cross-validation method should I, in general, use more often: k-fold cross-validation or repeated random sub-sampling validation.
From Wikipedia,...
- 121
0
votes
0
answers
7
views
Cascade Random Forest models or tuning?
I'm pretty new to Random Forests and Machine Learning in general, so if you see an alternative to my approach I'll appreciate any suggestions.
I want to create a model that classifies particles into 5 ...
- 1
8
votes
1
answer
212
views
What is the roadmap to self-taught probability and statistics for artificial intelligence?
I am trying to self-teach probability and statistics for Machine Learning career. However I want to learn very well as doing research in AI is my goal.
Which books should I use to learn probability, ...
0
votes
1
answer
16
views
HiClass: Modelling a Hierarchical Classifier
My question is specifically directed to the hiClass Python package for hierarchical classification (I am not sure if it is right to ask here, since I am not reporting an issue). After reading answer ...
- 253
3
votes
2
answers
332
views
Are we estimating the Bernoulli parameter in Logistic Regression?
In logistic regression, we often use maximum likelihood to estimate the parameter vector $\boldsymbol{\beta}$ that parametrizes the logistic equation.
My confusion stems from the following:
We know ...
- 715
-2
votes
1
answer
18
views
What does the error in artificial neural network stand for, is the same with mean square error (MSE) [closed]
How do I calculate mean square error (MSE) from the error obtained from ANN output
- 1
0
votes
0
answers
23
views
How to deal with Covid outlier in time series/machine learning forecasting?
Disclaimer: I checked some similar questions but I could not find anything in particular that would work for my case.
I am dealing with a time series going from 2015 to 2023. The data points are the ...
- 11
2
votes
1
answer
24
views
Machine learning method(s) to compare probability of success of two groups
I would like first to mention that I am relatively new to the Machine Learning (ML) world, but I have a decent background in statistics and econometrics. I am working on a research paper focusing on ...
- 21
0
votes
0
answers
6
views
Are some gradient weights equal?
I want to create a 3 layers neural network from scratch to perform linear regression.
The first and the second layer have 2 neurons, and the last layer has one neuron.
Feature vector x is divided into ...
- 1
0
votes
0
answers
23
views
In a predictive model, is orthogonality of noise and predictors an assumption or something that can be proved?
Consider the general problem of predicting the conditional mean $E(Y|X)$ where $X$ is the predictor. One assumes $Y$ can be written as:
$Y=f(x)+e$ where $E(e|X)=0$ which implies covariance of ...
- 129
0
votes
0
answers
25
views
2-class logistic regression on a 100x2 dataset [closed]
All in MatLab, Im trying to set the labels of the first 50 data samples to 1, and the second 50 samples to 0. Then there is some setup:
Use extended data vector xn = [1 xn]' for all data samples
...
- 1
0
votes
0
answers
6
views
How to predict multi-class multi-level category data in machine learning classification?
I have a data that it is include multi-level categories. I need to classification it by multi-label and multi-level classification model. may please tell me how can I do it by machine learning or ...
0
votes
1
answer
25
views
0
votes
0
answers
24
views
Missing at Random / Missing not at Random
I plan a study where I predict variables with machine learning methods integrating data from multiple surveys. The questionnaires assessed with each survey slightly differ, so that I have a lot of ...
0
votes
0
answers
17
views
Reduced Rank Linear Discriminant Analysis vs Fisher Discriminant as mentioned in Element of Statistical learning section 4.3.3
In ESL section 4.3.3 , Author gives three steps for finding optimal subspaces using LDA as below
compute the K × p matrix of class centroids M and the common covariance matrix W (for within-class ...
1
vote
0
answers
49
views
Show Neyman orthogonality condition for given score
The partially linear model, $\theta_0$ and $g_0,m_0 \in F\subseteq \{f|f:\mathbb R^d \rightarrow \mathbb R\}$ where $F$ is a fixed function class, and we consider a random vector $Z=(D,X,Y) \in \...
- 315
1
vote
0
answers
9
views
SVM, Is the slack value always equal to the alpha value for points within the margin?
I'm preparing for an exam and I got stuck on this question.
I understand that the alpha values 'affects' how much influence corresponding data point has on the position of the decision boundary, and ...
- 11
2
votes
1
answer
25
views
ROC AUC has $0.5$ as random performance. Does PR AUC have a similar notion?
In considering ROC AUC, there is a sense in which $0.5$ is the performance of a random model. Conveniently, this is true, no matter the data or the prior probability of class membership; the ROC AUC ...
- 46.4k