Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
65
votes
7answers
8k views

Variable selection for predictive modeling really needed in 2016?

This question has been asked on CV some yrs ago, it seems worth a repost in light of 1) order of magnitude better computing technology (e.g. parallel computing, HPC etc) and 2) newer techniques, e.g. [...
0
votes
0answers
6 views

Why deep Q learning works?

In DQN we evaluate true Q value by formula $$r + \gamma \max \hat Q\left( {s,a,{w_{{\rm{targe}}t}}} \right)$$ And use the output of Network to fit it. Why we can use this formula to approximate true ...
1
vote
2answers
20 views

What is the relation between “conjugate priors” and the approximate inference?

I know that "conjugate prior" is to help us calculate the the denominator of the Bayes formula(to make the calculations easier). And I just learnt to approximate the inference by mean field ...
1
vote
0answers
6 views

Maximization bias in Reinforcement learning

I have a question about Maximization biased in RL. In Richard S. Sutton and Andrew G. Barto 's book page 156 which says maximization biased occurs when estimate the value function while taking ...
0
votes
0answers
7 views

Is each row of latent factors obtained from matrix decomposition (SVD) dependent on the other rows of the higher dimensional matrix?

I implemented a recommendation system using user-user interaction data, learning missing ratings through alternating least squares and matrix factorization, which as I understand it, adjusts and ...
1
vote
3answers
341 views

XGboost for Time series - using lag of target variables

I'm trying to make a time series forecast using XGBoost. I have already added many time related variables - day_of_week, month, week_of_month, holiday. I want to add lagged values of target variable ...
0
votes
0answers
39 views

Predicted individual treatment effect with continous treatments

I'm trying to apply Rubin's counterfactual model in an observational setting using machine learning predictions to simulate the unseen treatment-outcome pairs, according to https://www.ncbi.nlm.nih....
0
votes
0answers
59 views

What does it mean if the sobol main and total effects indices are the same?

What does it mean when the total and main effects ANOVA indices are the same? Does it mean there is zero interaction of the different inputs? Is there some other way to quantify or understand that? I ...
1
vote
1answer
15 views

What does “knn is only approximated **locally**” mean?

Wiki gives this definition of KNN In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input ...
1
vote
1answer
45 views

How to recover primal problem from its dual counterpart

I am asking this from context of optimization in machine learning. We often talk about a primal problem and how this primal problem can be solved by first converting it into a dual problem (Using ...
1
vote
1answer
32 views

Remove seasonality for time series for regression

I want to perform a regression between 3 variables [x1,x2,x3] that have no trend and no seasonality across their time observations and a variable [Y] that has trend and seasonality. For [Y] I've ...
-1
votes
2answers
44 views

Clustering groups of data with Machine Learning

I want to cluster objects. There are two attributes (one categorial and one numerical). They should be clustered after the numerical attribut. But observations with the same categorical value should ...
2
votes
1answer
486 views

Planning VS Reinforcement Learning for Large State Spaces

Does knowing everything about your environment yield any major shortcuts to finding the optimal policy, in a Markov Decision Process with a very large (finite) number of states? Mere planning clearly ...
3
votes
1answer
58 views
+50

What is the “credit assignment” problem in Machine Learning and Deep Learning?

I was watching a very interesting video with Yoshua Bengio where he is brainstorming with his students. In this video they seem to make a distinction between "credit assignment" vs gradient descent vs ...
0
votes
2answers
46 views

Relationship between mean and variance of samples

I am thinking about the relationship between sample mean and variance in an example. If we want to look at the average goals per month for a soccer team. And we have mean and variance of goals for ...
0
votes
1answer
600 views

CNN convolutional layer backpropagation formulas

I tried to implement a CNN in Java but I am stuck at updating the weights in my convolution layer. I tried to create the following image that shows how I calculate each weight delta and error signals:...
1
vote
0answers
25 views

what is a parameter and hyperprameter [duplicate]

I hear in many articles the word parameters and hyperparameters but I don't know what they mean by that. Are they variable or the weights of the nodes? explain me in an intuitive way as an analogy ...
2
votes
2answers
154 views

Regression model when the dependent and independent variables show exponential distribution

As the Title suggests i am trying to figure out what would be the regression model to use when both the dependent and independent variables show an exponential distribution. Do I have to perform a ...
4
votes
1answer
1k views

Calculate the decision boundary for Quadratic Discriminant Analysis (QDA)

I am trying to find a solution to the decision boundary in QDA. The question was already asked and answered for LDA, and the solution provided by amoeba to compute this using the "standard Gaussian ...
1
vote
0answers
24 views

How to show an alternate data processing inequality concerning KL divergence between conditionals?

Suppose $(Y,X) \sim F \in \mathcal{P(\mathbb{R^d})}$. Consider an arbitrary transformation $f$ that acts on $X$. My intuition is that the following should be a result in information theory: $$ \...
3
votes
0answers
21 views

The extrapolation problem: model selection, performance metrics, and improvement

Machine learning models are fit to a response within a given range. This leads to weak and sometimes disastrous performance when it comes to instances outside that range. When the underlying mechanism ...
0
votes
0answers
20 views

predicting who will buy what [on hold]

what machine learning algorithm would be most suitable to predict which customer will buy what and when they will buy it ( consider historical data is available). I have tried : predicted next ...
0
votes
0answers
4 views

what is the cost function for a perceptron muticap

I have the function of cost or error of a perceptron of an entry and exit ...
2
votes
2answers
2k views

Naive Bayes: Mix unigrams and bigrams for text classification?

I'm creating a naive bayes text classifier, but I'm wondering if it's a good idea to break the text up into both unigrams and bigrams. Should I only use one method? Will having both variations mess ...
0
votes
0answers
33 views

VAE gives the same outputs [duplicate]

I'm trying to make a variational autoencoder with PyTorch that generates made-up pronounceable words. I'm following some tutorials and using this code as an example. The training data is the 10,000 ...
1
vote
0answers
86 views

Neural Network regression on time series

I want to predict the trend values of a time serie [Y] based on the effect of other 10 input variables which can also have interaction. Since the combination of interaction between the inputs is ...
0
votes
0answers
17 views

Keras predict probability too high [on hold]

I've followed this tutorial to create a simple neural network that matches intents. The results are good and the process is relatively fast, however there's an issue with it. The probability is ...
0
votes
2answers
22 views

What is the interpretation of the weights in the GMM?

GMM is $p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the interpretation of the weights here? Do ...
1
vote
1answer
513 views

Combining semantic segmentation with image classification (FCN + CNN)

I am currently working on a project that involves classifying each image as Good/Bad/Failed. We have a working convolutional neural network approach that works decent. I also have trained a Fully ...
-1
votes
1answer
49 views

Which is the correct method for outlier analysis on a dataset for modelling?

I'm trying to build a regression model but my data-set have many outliers points which I need to analyze and then remove them. There are two ways to do it, 1) First do all the analysis on every ...
135
votes
6answers
119k views

What are the advantages of ReLU over sigmoid function in deep neural networks?

The state of the art of non-linearity is to use rectified linear units (ReLU) instead of sigmoid function in deep neural network. What are the advantages? I know that training a network when ReLU is ...
0
votes
1answer
431 views

Use basis function transform non-linear to linear model

Left picture is showing that the vector data can not be separated by a linear line. but after You are using some basis function $\phi (x)$. it transforms to the right picture.I kind of understand the ...
0
votes
1answer
149 views

Truncated backpropagation for text

I know truncated backprop is useful for training RNNs like LSTM. But in some datasets of corpus, there are many long sequences with thousands of tokens. However the average length of tokens is just ...
0
votes
1answer
222 views

Tensor Classification Models

Aside from Convolution Neural Networks, are there any other methods that allow for classification of Tensors? My observations consist of multi-dimensional tensors with height of 1, where each channel ...
1
vote
0answers
394 views

How to calculate cohens kappa with multiple classes?

I'm working with an imbalanced data set including 12 classes. I was looking for a metric which I can minimize in my objective function during hyperparameter tuning and the final evaluation of the ...
1
vote
1answer
17 views

GBM: How to interpret relative variable influence

I recently used the gbm package in RStudio for my analyis. All worked well. But I struggle to understand the summary of the model. How to interpret the relative ...
0
votes
1answer
22 views

How do I predict future scenarios after training and validating my model?

Problem I'm new to machine learning and need a little activation energy to get me past this sticking point. I've trained/validated/tuned, and tested a random forest model. Therefore, I've used my ...
3
votes
1answer
37 views

Motivations for experiment design in statistical learning?

My interests in statistics centre around statistical learning, including Bayesian inference, inference in combinatorial spaces, Monte Carlo methods, Markov decision processes, modeling stochastic ...
0
votes
0answers
19 views

predicting future orders based on past ordering data [on hold]

I'm a student and as my project problem statement is as follows: a company has provided ordering data ( SKU, quantity,date of order, customer name) of past 3 years in form of an excel file. we want ...
0
votes
1answer
148 views

what are mixed variables in data mining?

I read that neural networks, SVM and neuro-fuzzy don't support "mixed variables." So what are those exactly? Does it refer to mixed types (numeric and non-numeric)? And if so, does that mean the ...
0
votes
1answer
39 views

Tuning SVM parameters in R

I am training an SVM model for the classification of the variable V19 within my dataset. I have done a pre-processing of the data, in particular I have used MICE to impute some missing data. Anyway a ...
0
votes
0answers
49 views

How to create a test set in stacking when doing cross validation

I am using Weka to implement stacking with k-fold cross validation. As I understand, we first divide our dataset in to k folds, then we use k-1 folds for training and 1 fold for testing. This ...
2
votes
2answers
322 views

Treating continuous variables

I attended a conference on ML and Data Science and I have a general question that was not answered in the conference. If we have a continuous variable, let's say age. What is the best way to handle ...
1
vote
0answers
74 views

Loss functions for Regression task

I am trying to understand the idea of Loss functions For Regression Task perfectly. I have read many textbooks and articles, and I came up with questions related to this subject. Several different ...
0
votes
1answer
149 views

Using gradient descent to train dual formulation of Kernel SVM

I've seen other posts about using gradient descent for the primal form, but not the dual form. In this book, the author discusses using (projected) gradient descent for the dual form: http://ciml....
2
votes
2answers
63 views

Consider this particular linear regression, $y=w_1x_1+w_2x_2$, is $R^2$ its hypothesis space?

Wikipedia gives this definition of the "hypothesis space" The hypothesis space is the space of functions the algorithm will search through. Consider this particular linear regression, $y=w_1x_1+...
0
votes
1answer
12 views

Is it necessary to scale the dependent variable in k-NN regression?

I want run kNN analysis to predict Y (continuous variable). I know that it is necessary to normalize all of the Xs. My question: is it also necessary to normalize Y values?
0
votes
1answer
1k views

How to make stochastic gradient descent algorithm converge to the optimum?

(Background info taken from my blog) In logistic regression, the hypothesis function, which models the relationshiop between the dependent variable $P(y = 1)$ and the independent variable $X$, is : ...
1
vote
1answer
151 views

Intrinsic dimensionality and density-based clustering

I’ve got several thousand observations in 350-dimensional space, in a relatively sparse matrix (median observation has 11 non-zero dimensions). I'm using a density-based clustering algorithm, DBSCAN, ...
0
votes
0answers
11 views

Random fourier features and Bochner's Theorem

The paper, Random Fourier Features for Large-Scale Kernel Machines by Ali Rahimi and Ben Recht , makes use of Bochner's theorem which says that the Fourier transform $p(w) $ of shift-invariant ...