Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Repeated K fold cross validation

I want to use repeated Kfold cross validation in my experiment, since I have a samll dataset that might be prone to fluctuation in results of a regular cross valudation regime, so I am opting for a ...
0
votes
0answers
8 views

How can we conclude that an optimization algorithm is better than another one for a problem at hand

When we test a new optimization algorithm for a particular problem at hand, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,...
3
votes
3answers
322 views

Neural Network vs regression

I have a small numeric dataset with 20 observations and 30 variables. I want to predict Y as a function of the rest 29 Xs(x1,x2,x3...x29). I've tested: neural network (NN) with 1 hidden layer and 7 ...
0
votes
0answers
9 views

Estimate the density of a random variable purely from another dependent random variable?

For example, if we want to estimate the density of a binary random variable $Y\in\{-1,1\}$ (such as binary classification), it is possible to do that by purely observing a dependent variable (such as ...
0
votes
1answer
380 views

On the convergence of Iterated Conditional Modes (ICM) for MAP inference

ICM is very fast but I could not find any references that contain a detailed analysis on its convergence (e.g. rate of convergence). Any suggestions please? Thanks a lot for your help!
1
vote
2answers
139 views

What should be the optimum number of features for 10 million observations for kmeans clustering?

I have a dataset of 10 million observations and 100 million features. I have to perform kmeans clustering on that dataset. The approx value of k is 30000 Is it advisable to perform clustering with ...
0
votes
0answers
10 views

How to prove if h is PAC learnable?

Let $X = \{(x_1, x_2) : 0 \leq x_1 \leq 1, 0 \leq x_2 \leq 1\}$ be the two-dimensional unit square. Let $H_0$ consist of all sets consisting of a finite number of points from $X$ . Let $H$ contain ...
4
votes
0answers
17 views

Workaround for word embeddings that do not “see” antonyms

Most word embeddings do not "see" antonyms. For instance, among many words they will place vectors for "dependent" and "independent" (as an example) quite close, - actually as close as with synonyms ...
0
votes
2answers
142 views

Compute standard deviation of accuracy

edit - more information about what the code given should represent The following pseudocode outlines the problem as I have it ...
0
votes
0answers
6 views

What is best way to crossover[Genetic Algorithms]

In Genetic Algorithms there are five phases 1)Initial population 2)Fitness function 3)Selection 4)Crossover 5)Mutation I have solved 2-dimentional problem using this algorithm ...
0
votes
1answer
13 views

Is it wise to reduce the number of labels in a multi-class classification problem?

I'm working on a dataset which has 5 labels or the outcome, Y. I'm going to use ML model to predict the 5 labels. While doing the data analysis, I found that class1(60%), class2(39%),class3(0.33%), ...
0
votes
0answers
1k views

SSD MobileNet v1 loss not converging bounding boxes all over the place

I've trained SSD MobileNet v2 model using Tensorflow API on my own dataset of ~4k dog pictures and it displays bounding boxes all over the place. I've trained with batch size 1. The same dataset ...
1
vote
1answer
30 views

Linear Regression: how to discern a possible correlation between observation errors from scatter plot

The following is excerpted from An Introduction to Statistical Learning by Tibshiriani et. al. In page 66, they introduce the standard error for linear regression coefficients: $$Y = \beta_0+\...
2
votes
2answers
57 views

Why does this paper use training and test sets with different distributions?

From Doruk Cengiz. Seeing beyond the trees: Using machine learning to estimate the impact of minimum wages on affected individuals. 2019. I noticed a strange way of building the training and test ...
4
votes
5answers
10k views

Cost function turning into nan after a certain number of iterations

I have a question and would like to hear what the community has to say. Suppose you are training a deep learning neural network. The implementation details are not relevant for my question. I know ...
0
votes
1answer
274 views

How do I optimize decision tree regression algorithm implemented in R?

I'm only getting an accuracy of 59% using the following implementation calculated using the diag(sum(cm)) and sum(cm) functions. ...
42
votes
3answers
29k views

Understanding the role of the discount factor in reinforcement learning

I'm teaching myself about reinforcement learning, and trying to understand the concept of discounted reward. So the reward is necessary to tell the system which state-action pairs are good, and which ...
15
votes
3answers
9k views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
1
vote
2answers
38 views

Splitting event data into training and testing with all the events in training ending before the events in testing starts

I have a dataset with event data. It has a date of start and date of finish vareiable. I need to predict time remaining until an event finishes. The problem is that I can't use events in future time ...
-1
votes
1answer
102 views

Is time series analysis suitable for my dataset?

I am monitoring user behavior, while the user interacts with a form on a website. That form has multiple textfields from top to bottom and at the bottom it has two buttons: "cancel" and "save". My ...
9
votes
1answer
2k views

Complex analysis, Functional analysis for deeper understanding Machine Learning

I want to get deeper into the Machine Learning(theory and application in Finance). I want to ask how relevant are complex analysis and functional analysis as a basis for Machine Learning? Do I need to ...
1
vote
2answers
136 views

Heterogeneous Domain Adaptation without training data from target domain

Are there any strategies to learn a model that can classify data from one domain using only data from a different domain for training? For example, suppose I have a bunch of data from two different ...
0
votes
0answers
29 views

Splitting “Changing” Multivariate Time Series Data

Background I am working on a project involving a large multivariate time series dataset (1m obs, 60 col, 1 target). The target appears to be stationary when tested and plotted. I have split the ...
0
votes
1answer
171 views

Label smoothing formula

I recently came across this paper in section 3.2 it talks about label smoothing loss and how it's equivalent to s equivalent to adding the KL divergence between the uniform distribution u and the ...
2
votes
1answer
447 views

Multiple Response Regression in Spark MLLib

I am trying to do a regression using RandomForests in Spark ML where I have several input variables and would like to predict several responses. Training data would look like ...
2
votes
1answer
176 views

Is the absolute value of the difference a kernel?

In particular is $$ k(x_i,x_j)=|x_i-x_j|, \quad x_i,x_j\in \mathbb{R}$$ a valid kernel?
2
votes
0answers
192 views

What is the most intuitive proof that Gaussian kernel is positive definite? [duplicate]

I have general form of Gaussian kernel $K(x,x')=\exp(-\|x-x'\|^{2})$ (just not considering $\sigma$). I tried to prove its positive definiteness via Gram matrix properties, but couldn't. Is there any ...
2
votes
1answer
52 views

What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?

What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?
6
votes
1answer
367 views

Unsupervised pre-training for Reinforcement Learning

Since the advent of many unsupervised learning methods, as a pretraining step for the main supervised task (mostly under the name of Deep Learning), it shouldn't be strange to ask, what is the current ...
0
votes
0answers
13 views

Sample complexity of mean estimation using empirical estimator and median-of-means estimator?

Given a random variable $X$ with unknown mean $\mu$ and variance $\sigma^2$, we want to produce an estimate $\hat{\mu}$ based on $n$ i.i.d. samples from $X$ such that $\rvert \hat{\mu} - \mu \lvert \...
0
votes
0answers
11 views

Do I have to use MinMax Scaling? [on hold]

I'm curious that in sklearn KNN already use MinMax inside the code or do I have to scale my data before put it in sklearn knn?
4
votes
1answer
503 views

How Variance becomes infinite [on hold]

The text is from Introduction to statistical learning, by James et al., page 203-204 The doubts are highlighted in bold. Please help me to understand this. p-No. of Independent variables , n-No.of ...
0
votes
0answers
15 views

How to test significance of difference between regression coefficients for multiple interaction categories?

Suppose I have N multiple categories (a discrete interaction variable) and representative samples of which. I'm fitting a multi-variate X linear regression model to response variable Y for each ...
0
votes
0answers
10 views

chi-square test of the log likelihood ratios for a nested hypothesis testing on 4 models of linear regression

I trying to reconstruct a procedure from an article that I read. First, I had to build 4 models to my raw data (points of X and Y)- one model is constant, second is linear, third is constant and then ...
7
votes
2answers
1k views

How to propagate uncertainty into the prediction of a neural network?

I have inputs $x_1\ldots x_n$ that have known $1\sigma$ uncertainties $\epsilon_1 \ldots \epsilon_n$. I am using them to predict outputs $y_1 \ldots y_m$ on a trained neural network. How can I obtain ...
0
votes
1answer
17 views

Nested Cross-Validation on the whole data?

I am performing nested cross-validation, and I know that the idea behind it is to see how the model generalizes. For that, we don't only shuffle the training data but we also do shuffle the testing ...
2
votes
1answer
42 views

“Learning curve” behavior comparison

I am comparing two learning algorithms using the log learning rate. I plot training data size vs. Mean Absolute Error (MAE). In the figure below, you can see method 1 shown with black line and method ...
0
votes
0answers
15 views

Neural network for PDE: Should we train the PDE using more initial and boundary data at the beginning?

I was trying to solve a partial differential equation (PDE) using a neural network. The solution to the PDE is not unique unless the boundary condition is determined. In my case, the neural network ...
0
votes
1answer
30 views

Variational inference with dependent variables

Classical variational inference uses mean field theory because of its computational benefits, i.e. assumes that the latent variables are independent. For gaussian distribution, it wants to find a ...
1
vote
2answers
41 views

How to deal with biased dataset for both training and testing data?

I am currently working on a classification problem with a highly biased dataset. The dataset is biased for both training and testing data. And I am having trouble dealing with the dataset or modifying ...
0
votes
0answers
15 views

Polynomial regression on uncorrelated features

I'm looking to do polynomial regression on 2 relatively uncorrelated features. Even with very high degrees, I find that the model doesn't fit very well to the data (figure attached here). Since the ...
0
votes
0answers
9 views

Learning to rank for time series with LightGBM data formatting

I have multiple time series, one for each item, say item1, item2,...itemN. N>=500. I have features associated for each of the items and a dependent variable. I am currently studying learning to rank ...
1
vote
1answer
508 views

machine learning algorithms (Xgb, LSTM, others) for time series forecasting

I have seen many kernels that are using machine learning algorithms (Xgb, LSTM, others) on time series forecasting. A time series data typically has trend and seasonal components. In general my ...
2
votes
2answers
182 views

Regression model when the dependent and independent variables show exponential distribution

As the Title suggests i am trying to figure out what would be the regression model to use when both the dependent and independent variables show an exponential distribution. Do I have to perform a ...
3
votes
1answer
48 views

How interaction terms are treated in Gradient Boosting?

In GAMs interaction terms have to be expressly specified as covariates, even for simple linear relationships. On the contrary, with Gradient boosting this is not nesessary because the algorithm itself ...
21
votes
1answer
2k views

Choosing among proper scoring rules

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
1
vote
1answer
796 views

What is minibatch discrimination?

Can someone explain what minibatch discrimination is, in simple terms? Here is the link to the original paper. (Minibatch discrimination: Sec 3.2, page 3)
0
votes
1answer
58 views

Could someone give an concrete example to illustrate how “empirical distribution” relates to “histogram”?

This question is derived from this one, which is related to empirical distribution. I did a little bit search and then got this and this, unfortunately, none of them mentions "histogram". I've ...
2
votes
1answer
204 views

classification on imbalanced dataset via random forest: results vary with random seed

I have a highly imbalanced dataset of about 8000 observations, with 11 features and one binary target variable. I want to predict the target labels, considering that the "1" target label occurs for 1....
4
votes
2answers
3k views

multiclass classification having class imbalance with Gradient Boosting Classifier

I am using Abalon data for classification from UCI(https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data). I have scaled data and used TSNE for visualization. ...