Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Methods and principles of building "computer systems that try to automatically improve with experience."

0
votes
0answers
5 views

Feature selection for categorical data

I am trying to apply the classification tree on a dataset that have mainly categorical data. I have some doubts about the label encoding and feature selection. What are the steps that I have to follow?...
3
votes
1answer
25 views

Redundant variables in linear regression

If I have some number of independent variables, and one dependent variable, and some of those independent variables are strongly correlated with each other, does that make one of them redundant?
0
votes
0answers
5 views

Synthetic population generation | Data aggregated at postal code level

Folks, I am trying to generate synthetic population from data aggregated at a postal code level. Most resources I found on the internet enables users generate synthetic population from house-hold ...
0
votes
0answers
12 views

In what sense is momentum “optimal” for optimisation of neural networks?

This article states about the use of momentum in gradient descent for neural networks: A lower bound, courtesy of Nesterov [5], states that momentum is, in a certain very narrow and technical sense,...
1
vote
1answer
24 views

Why are the trivial points included when calculating AUC?

I'm aware of some of the issues associated with using AUC for model comparison (see for example the articles referenced on Wikipedia: here, here, or here). But so far I have found nothing on an issue ...
1
vote
0answers
9 views

How do you cross-validate moving time predictions?

I have a model that trains itself by looking at the last 12 months of data, and then predicts the next month (out of sample). Say I have 24 months worth of data, thus allowing me 12 opportunities to ...
0
votes
0answers
5 views

Why is the second derivative of cost w.r.t. parameters higher in deeper layers in NN?

In the paper "[Efficient BackProp][1]" , the authors state: The second derivative of the cost function w.r.t. weights in the lower layer is generally smaller than that of the higher layers. What ...
0
votes
0answers
6 views

Classification confidence at Multi-class classifier with Soft-max output

what I'm NOT asking about is : Confidence interval So the question I'm trying to find the answer of would be : how to measure the confidence of a multi-class classifier with a Softmax output (eg a ...
1
vote
0answers
5 views

How to handle variable size input data (incomplete) to build/train a NN for regression?

Suppose you have the classical example of predicting house prices and you have as input features area size, built year, number of previous owners, city, number of floors, number of bedrooms, etc. But ...
1
vote
0answers
15 views

What does “learn the linear part of a mapping” mean?

In the paper "Efficient BackProp" , the authors talk about initializing the weights not too small and not too large: Intermediate weights that range over the sigmoid's linear region have the ...
0
votes
1answer
13 views

How to categorize data as others if training set is not available?

I run into a problem. I am using the decision tree to classify the incident category based on the short description the user has used while logging the ticket. I have the training data only for 5 ...
2
votes
1answer
36 views

Does taxi demand follow a poisson distribution?

The important assumptions underlying a Poisson process are: What happens in one subinterval of time is independent of what happens in any other subinterval The probability of an event is the same in ...
0
votes
1answer
31 views

Question regarding machine learning models in production

for example, i have a feature with 5 distinct values and once one hot encoded this becomes 5 columns, but lets say the data that needs to be predicted has 4 distinct values, the neural network won't ...
1
vote
1answer
29 views

How is it possible? Training and validation loss curves were decreasing while training data size was increasing

I'm really puzzled... I’ve learned and observed that training loss / error increases with training data size as stated in Dr Andrew Ng’s ML course. I’ve recently experienced an anomaly. Training loss ...
0
votes
0answers
4 views

ML Regression: Rounding up -ve values for predictions to 0 (if -ve outcomes are impossible) before testing algorithm accuracy?

I am working on a Machine Learning linear regression problem where the output cannot be negative. However when I am running my learning algorithm, the predictions for low values in the test data tend ...
0
votes
0answers
7 views

One Hot Encoding: split feature into as many categories as possible, or lump data into smaller no of bins (including multiple split categories)?

I am working on a Machine Learning problem on a bike-share system database to predict the total number of bikes rented (per hour) based on other data. I used one-hot-encoding to split up the ...
0
votes
0answers
6 views

Rules based Model (Function) - Derive Probability & Ensembling

Basically, let's assume I have a simple rules-based function/model (if weight >= 150) -> return true. Simple binary answer (true or false) from a single feature input. If I have a range of samples/...
0
votes
0answers
7 views

Can I use Gradient Boosting Classier to determine feature importance without worrying about precision, recall and accuracy?

My boss is interested in understanding how certain actions improve user retention WoW. I decided to build a GBDT model to assess those features. My question is: Does accuracy, precision or recall ...
1
vote
0answers
22 views

How to model and estimate interference and subsequent shocks on panel data?

I have the following setting: In my factory, we have mutliple assembly lines(>10). Each line produces an amout of itmes every day, with some weekly and mothly production peeks. Thus its basically a ...
0
votes
1answer
14 views

VC dimension of Parity Function

Reference: Machine Learning Foundations lecture by Yishay Mansour. I am struggling to understand the explanation when they arrive at $X_S(e_j)$ Could someone provide some details? Thanks.
0
votes
0answers
9 views

Self Attention in the Transformer learning algorithm

in this article can somebody tell me where the heck the Wq, Wk, and Wv matrices in the “Self-Attention in Detail” section come from a little more intuitively and specifically since the article doesn't ...
0
votes
1answer
16 views

School Dropout Prediction

I have a dataset composed by several features group by some factors (academic, personal, economic). I would like to predict the risk (high, medium, low) of dropout and its respective risk percentage. ...
0
votes
0answers
6 views

Testing model trained on Standardized/Scaled data

I have a dataset that was passed to a StandardScaler before being passed into a classification model for training. StandardScaler was also applied to the test data for Model validation. But, the ...
0
votes
0answers
8 views

Aggregation across sentences in a document

I'm using deepMoji for a text classification problem. Deepmoji returns a vector of 64 emoji for twitter sized text. I'm running deepMoji on documents that are much longer and I'm wondering if there ...
0
votes
0answers
11 views

Treating multilabel as multiclass perfomance

I am struggling to comprehend something. In case of several labels prediction, when multiclass prediction should be prefered over the multiple multi-label predictions? Assume that we have 10 binary ...
0
votes
0answers
16 views

how to choose model when training accuracy is lower than validation accuracy while training neural network?

Below is a specific case but a general situation i find myself involved with in my job. This question is intended at getting ideas on how to pick the best model: Dataset: rows: 10,166, features: ...
1
vote
0answers
18 views

xgboost with 3500 features

I'm trying to make a solution for: https://www.kaggle.com/c/two-sigma-financial-news/ This question relates to using only market price data to predict future prices (i.e. not news data, as specified ...
1
vote
0answers
10 views

Linear Regression - Which Features Should We Apply a Polynomial Transform to and Why? [on hold]

In which situations would a feature have a polynomial transformation appropriately applied to it, and why would we do this; what ultimate impact does this have on the data. Supposing we select the ...
2
votes
1answer
24 views

Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID

From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
0answers
10 views

High variance of returns using Asynchronous Actor-Critic Agents (A3C) on CartPole [on hold]

I ran the code from the Tensorflow blog with modified running average function (it takes a running mean of the last 3 episodes only) and notice strange behavior. Although the model achieves episode ...
0
votes
0answers
24 views

Best fit model in R [on hold]

My predictor variable(x) and response variable (y) are as the following. I tried fitting using multi linear regression, polynomial regression etc. I tried removing the influential points found by cook'...
0
votes
0answers
18 views

Bellmans Equation in Sutton and Barto

If you haven't looked at Sutton before please ignore this question, I have not explained every aspect of the notation I've always been a bit confused by the derivation of Bellman equations for the ...
0
votes
1answer
21 views

How can we interpret the learning curve including loss for training and test in a deep learning model?

I am working on 3D medical image segmentation area. It may take 2-3 days to finish one round of training. How can I interpret the learning curve if over-fitting is happening or not? It happens to me ...
1
vote
2answers
34 views

In Recurrent NN, what's the reason for adding instead of multiplying the input term and the state term in the hidden units?

As we know, the hidden layer unit has the following activation: $$h_t=tanh(UX_t+Wh_{t-1})$$ So there is the interaction between the input and the previous state: $UX_t+Wh_{t-1}$. My question is why it ...
0
votes
0answers
19 views

Curse of dimensionality after using dummies

TL;DR : do you really need n>d^2 samples after using dummies ? I needed to do a decision tree for data that is mostly string-ish (sklearn) , so I used binary encoding (e.g dummy variables) and from ...
1
vote
2answers
27 views

Mathematical structure of SimpleRNN in keras

Two types of RNN can be used: Type1: The output is being used as state h(t) = g(W1.x(t) + W2.h(t-1) + b1) Type2: There is a state in addition to the output a(t) = g(W1.x(t) + W2.a(t-1) + b1) h(t) ...
0
votes
0answers
9 views

(Why) Is the ouput's dimension in a LSTM bound to the number of recurrent units, and how are recurrent outputs passed?

Background: Looking for specific/interesting information on the equations within LSTM-Networks, I found the paper LSTM: A Search Space Odyssey. It is frequently mentioned in other articles. To gain a ...
0
votes
0answers
15 views

Standard Notation For Unknown Parameters

I'm currently going through K. P. Murphy's "Machine Learning: A Probabilistic Perspective". The following notation is used throughout the book: $$p(y = c|x, θ)$$ to denote a generative classifier and $...
0
votes
1answer
13 views

Classification datasets where Gradient Boosted Trees are consistently outperformed (in terms of simple accuracy on test set) by other models?

I'm looking for datasets that are known to be especially difficult for GBTs and I'm not sure where to look without needing to evaluate the model on them myself. Thanks!
1
vote
0answers
14 views

Prove that given kernel is valid and find the relevant mapping

Understanding Machine Learning: From Theory to Algorithms, Section 16.6, Question 4 is For $x>z$, I formulate my kernel matrix as $K = [x \quad z;z \quad z]$ which gives the cofactors as $x, z(x-...
0
votes
0answers
6 views

How can stratified kfolds perform worse than regular kfolds?

I am working with unbalanced classes to solve a classification problem (whether individuals pay their fees or not). My class imbalance is 75% positive (paid) and 25% negative (unpaid). I have read ...
0
votes
0answers
16 views

Proving a given kernel is not valid [duplicate]

$K(x,z) = e^{\gamma ||x-z||^2} \quad \gamma >0$ I took $x_1 = [1\quad 0]^T$ and $x_2 = [0\quad 2]^T$ which gives $K(x_1,x_2) = 5 = K(x_2,x_1)$ and $K(x_1,x_1) = K(x_2,x_2) = 0$ Thus the Kernel ...
0
votes
0answers
9 views

Why my loss function visualization reduce not stable? [on hold]

I train CRNN (combine CNN with RNN) for OCR. And I visualize it on test set. I see picture: What is my problem ? and What are the ways to solve it ?
3
votes
3answers
85 views

Are there any supervised learning methods that do NOT boil down to optimizing a loss function?

All of supervised learning methods I can think of amount to optimizing a loss function (RMSE, AIC, Cross-Entropy,...) against a labeled data set. One would think that "learning = optimizing loss ...
0
votes
0answers
33 views

Handling imbalanced data for classification [duplicate]

What are the best ways to deal with imbalanced datasets for classifying whether or not individuals pay their tuition? The data is 75% positive class (paid) and 25% negative (unpaid). Some approaches I ...
0
votes
0answers
15 views

VC dimension of signed intervals

Understanding Machine Learning: From Theory to Algorithms, Section 6.8, Question 9 is Let $\mathcal H$ be the class of signed intervals, that is, $\mathcal H = \{ h_{a,b,s} : a \le b, s \in \{-1, ...
0
votes
0answers
9 views

SVM Regression n R

Hi The following graph represents original data, linear regression and Support vector regression. I would like to know if this is a decent plot of SVR and not understanding how to predict for a new ...
2
votes
0answers
26 views

Best practices for dealing with missing data [on hold]

There are a lot of threads on here about missing data, but I haven't found something that really gets at the best practices, and discussion of why to choose one approach over another. This is such a ...
0
votes
0answers
7 views

Contradiction between accuracy obtained from my pretrained conv base network and pretrained conv base network in Deep Learning with Python

I trained a pretrained convnet model on the cats and dogs dataset and the following are the accuracies obtained: Freezed Conv Base ~ 90% Unfreezed Conv Base ~ 96% However this is in contradiction to ...
1
vote
1answer
25 views

Using categorical feature as both a continuous feature, and also doing One hot encoding. Is this overkill?

I am working on a Machine Learning regression problem, with a data-set where I have data from a period of several years. From the "date" feature, I extracted the week number (0-53). Next I am doing 2 ...