The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
0
votes
4answers
43 views

Can least-squares linear regression ever produce no solution at all?

Is it ever possible for least-squares linear regression (linear in both features and weights) NOT to produce a solution? That is, after we set each partial derivative to zero, can the resulting system ...
0
votes
1answer
268 views

Binary classification using GPML toolbox;

Using the demo given in the demo_classification file, I am trying to do a binary classification where each of my class contains 10 samples of 73 dimensions. Following is the code where I try to '...
0
votes
0answers
4 views

difference between RBF kernel ridge and gaussian kernel regression

This may seem a very naif question: theory frameworks behind kernel ridge regression and classic, non-parametric, kernel regression are very different, but still, from a practical point of view, I can'...
0
votes
1answer
21 views

Help understanding Vanishing and Exploding Gradients

I am following deeplearning.ai's videos on Coursera. I have a couple of questions regarding vanishing and exploding gradients. The following is Prof Andrew Ngs lecture slides: From what Prof Ng says ...
0
votes
0answers
32 views
+50

How XGBoost use Histogram to Determine Splitting Features?

I'm trying for some time to understanding, the splitting method used in XGBoost to determine the best split. But unfortunately I didn't find any clear explanation of it. I found this post which ...
7
votes
0answers
128 views

Random variables $X, Y$ such that $X$, $Y$ and $\sqrt{X + \sqrt{Y}}$ belongs to the same family of distributions?

Is there a family of positive distributions such that if $X$ has the distribution in question, then $\sqrt{X}$ also has a distribution from the same family. Ideally, it would be great if $X+Y$ also ...
3
votes
0answers
35 views

Does Bias-Variance Tradeoff always exist?

I'm following deeplearning.ai's videos on Coursera. In one of the videos, Prof Ng mentions: So a couple of points to notice. First is that, depending on whether you have high bias or high variance, ...
0
votes
2answers
433 views

What algorithm should I use to predict a continuous dependent variable from multiple continuous & categorical independent variables?

I'm software engineer of an E-commerce company, facing a problem like this: An e-commerce shop sells their products daily and wants to know what conditions that might improve their sales. I'm ...
1
vote
1answer
247 views

Estimating conditional probability with many samples

I am confused about the estimation of conditional probabilities. Suppose I want to predict a binary outcome variable $Y = 0,1$ given $n$ categorical features $X = (X_1, \ldots, X_n)$, i.e. to ...
0
votes
1answer
273 views

When doing Cox regression, what should we take as event duration for censored data, and how should we divide the data into training and testing sets?

I am starting out with survival analysis, and am confused about some things specifically in Cox regression. I have not found a tutorial that explains these things clearly, that's why I'm posting this ...
13
votes
2answers
6k views

What is “baseline” in precision recall curve

I'm trying to understand precision recall curve, I understand what precision and recall are but the thing I don't understand is the "baseline" value. I was reading this link https://classeval....
1
vote
1answer
136 views

Independent and dependent variables Machine Learning

I want to start a regression model on lottery numbers, my database consist of the dates, and results(numbers). In order to do a regression model I believe you need independent and dependent variables, ...
0
votes
0answers
8 views

Why is the stochastic gradient of a layer almost orthogonal to its weight?

In the paper Fixup Initialization: Residual Learning Without Normalization. In Page 5 when talking about effects of multipliers, the authors mentioned that Specifically, as the stochastic gradient ...
1
vote
2answers
126 views

Score of importance from feature selection techniques

Can I get the score of importance for each feature in feature selection methos such as Chi2, Information Gain (IG), or Recursive Feature Elimination (RFE)? Or they just provide a list of important ...
3
votes
1answer
272 views

Is it possible for a reinforcement learning agent to create or generate additional features

Based on what I've read, the best model-free reinforcement learning algorithm to this date is Q-Learning, where each state, action pair in the agent's world is given a Q-value, and at each state the ...
1
vote
2answers
509 views

Extending the idea of Bootstrapping to Train Test splits of a Dataset used to learn a Classifier in Machine Learning

In Machine Learning the standard practice for learning a Classifier --e.g. fitting a Logistic Regression model-- and then validating its performance is to split the original/available Dataset into a ...
0
votes
0answers
21 views

What is the intuition behind pi in the PDF of a Normal Distribution ? Is it related to some sort to a circle / sphere

The PDF of a Normal distribution is given as below I am aware of the various properties of Normal distribution and how the two parameters mu and sigma affect the shape of the distribution. What is ...
0
votes
0answers
14 views

Classifier output has 3 distinct peaks. Why? [on hold]

I trained a classifier on this dataset from r/ChangeMyView (https://chenhaot.com/pages/changemyview.html) on individual comments to predict whether or not that ...
0
votes
2answers
92 views

RBF kernel mapping

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere. Our RBF kernel given by: $k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$ Can anyone explain why ...
0
votes
0answers
8 views

Oversampling methods for numerical data (regression)

There are many oversampling methods for categorical labels (for example SMOTE and Rose, etc.). But, are there oversampling method for numerical labels (the thing that I want to predict with my ...
1
vote
2answers
46 views

How can I get the optimal perturbation of a trained model?

I get stuck while reading Goodfellow's paper on adversarial networks. In the explanation of the Figure 2 he stated that: b) The sign of the weights of a logistic regression model trained on ...
3
votes
1answer
22 views

About the need of splitting data in stacking

I learned stacking of machine learning in a book, hands-on machine learning 2nd edition (2019). The picture was cited from hands-on machine learning 2nd edition (2019). In the above situation, ...
1
vote
1answer
26 views

How is Akaike Information Criterion related to Information theory?

How is the Akaike Information Criterion (AIC) related to Information theory ? I mean from the equation (below), it is not at all intuitive how information theory comes into picture. Also is AIC ...
1
vote
1answer
220 views

Best apparoaches for feature selection in multilabel classification

I have dataset which consists of around 46k observations and 20k features. The target vector is of length 75 (and so the target matrix is 46k x 75). among the features few are categorical and others ...
0
votes
0answers
24 views

Bayesian Optimization

I am reading a paper on optimization and came across the following formula: Now my questions are: Question 1: Does the integral of the expression inside the means the expected value $\mathcal y_{t,...
2
votes
3answers
5k views

How to build a prediction model for exam score based on previous scores

I am trying to construct a formula, which will take student's previous exam results (for ex: SAT) taken at particular dates and predict his future test result. One X is previous test result 1; ...
1
vote
1answer
38 views

What is the relationship between estimation error, approximation error, bias, variance in machine learning?

I'm a beginner in machine learning. I was reading http://ciml.info/ 5.9 Bias/Variance Trade-off According to this book: The trade-off between estimation error and approximation error is often called ...
7
votes
3answers
760 views

Gradient descent of $f(w)=\frac12w^TAw-b^Tw$ viewed in the space of Eigenvectors of $A$

I'm reading Why Momentum Really Works, a post from the new distill journal. I'll paraphrase the main equations leading to the part which confuses me, the post describes the intuition in more detail. ...
0
votes
0answers
16 views

Is doing oversampling on train set and undersampling on test set correct?

I have an imbalanced dataset (95% in class 0 and 5% in class 1) and I am using machine learning for classification. The AUC(Area under ROC curve) was high (about 0.86) but AUPRC(Area under precision-...
1
vote
1answer
62 views

Neural Networks interaction between input variables

I want to build a regression NN model that predicts Y variable but takes into account the interactions between input variables (x1,x2,x3,x4,x5) without explicitly specifying them. My current NN model ...
0
votes
0answers
5 views

Modelling Technique for Cross Sell, Up Sell and Recommendation [on hold]

I am new to machine learning and exploring statistical techniques to Cross-Sell, Up-Sell and Recommend Life insurance to customers. I have transactional data of customers and their policy details. ...
3
votes
1answer
256 views

In a Boltzmann machine, why isn't there a simple expression for the optimal edge weights in terms of correlations between variables?

Suppose I have a fully connected, fully visible Boltzmann machine (no hidden variables) with binary variables $x_i\in \{+1, -1\}$ that defines the probability distribution $$ p(\mathbf{x} ; \mathbf{...
0
votes
0answers
14 views

Predicting multiclass outcome using R

The dataset contains about 17000 observations, the outcome variable (with about 10 classes) and the predictors (about 10) are all categorical variables. Which algorithm is the best to employ to train ...
1
vote
1answer
69 views

Two armed bandit with a known expectation

Assume a two armed case with bernoulli rewards. We know that UCB1 gives a pretty tight bound for multiarmed bandit cases. What if we know the mean of one arm, how can we obtain a better strategy/...
0
votes
1answer
13 views

What is the intuition behind the fact that the Explore Then Commit algorithm in a multiarmed bandit problem can achieve sublinear regret?

The thing that confuses me is as follows: no matter how many times we explore each arm at the beginning, there is some chance that the arm that performed the best on the sample is actually a ...
2
votes
1answer
41 views

Using IPS(inverse probability weighting) with a deterministic policy as the logging policy

In a contextual bandit problem, why can't we use inverse probability weighting (inverse propensity score) with a deterministic policy as the logging policy? Could you give me a concrete example?
57
votes
10answers
91k views

Validation Error less than training error?

I found two questions here and here about this issue but there is no obvious answer or explanation yet.I enforce the same problem where the validation error is less than training error in my ...
1
vote
0answers
18 views

Help, my neural network is only recognising one particular category? [duplicate]

I am very new to Neural network and python and only know about very basic stuff. I tried to train the neural network with four categories of data however, when tested, the neural network tend to get ...
3
votes
1answer
2k views

Is the Laplace/Lidstone smoothing parameter (talking about Multinomial/Bernoulli Naive Bayes) related to the particular structure of the dataset?

I'm working with Multinomial and Bernoulli Naive Bayes implementation of scikit-learn (python) for text classification. I'm using the 20_newsgroups dataset. From the scikit documentation we have: <...
0
votes
0answers
13 views

Combining features linearly before non-linear modelling

I want to construct a model like this: $$y = g(X, Z)$$ $$X = \alpha_1 x_1 + \alpha_2 x_2 + \dots + \alpha_n x_n$$ $$Z = \beta_1 z_1 + \beta_2 z_2 + \dots + \beta_n z_n$$ Where $x_i \text{ and } z_i$ ...
0
votes
0answers
9 views

Hard classification problem - test set accuracy seems to cap out at a certain value regardless of architecture [duplicate]

I've been working on a hard binary classification problem (50-50 split between classes). I've tried a variety of different network architectures and training schemes - dropout, no dropout, batch ...
0
votes
0answers
6 views

How to reshape input data in RNN model for prediction [on hold]

While making prediction with a built model using new data, an error returned because the shape is different: Here's the code: ...
0
votes
1answer
30 views

Matching vs simple regression for causal inference?

This is a really simple, newbie question. I am really confused about the notion of matching and when it can be used instead of a multiple regression? Assume I have listed all the confounding ...
0
votes
0answers
11 views

Including ratios as features in machine learning algorithms [duplicate]

Assume that body mass index (BMI) is a good predictor for early death which we are trying to model using a host of different algorithms. A colleague posed the following to me: would it be better to ...
1
vote
0answers
14 views

Land Method for Lognormal Distribution

The Land Method is explain here. Anyone knows how I can look up the H*1-α* It says "Tables of these values are given by Gilbert (1987, Tables A-10 and A-12) and Land (1975). " Gilbert (1987) is not ...
0
votes
0answers
21 views

Is term “metric” for evaluating machine learning model misnomer?

Term "metric" is used in many popular machine learning articles [1, 2] for describing an evaluation criterium of the performance of a model. Although, mathematically is the term defined as: In ...
0
votes
0answers
8 views

Feature Extraction, Moving window approach [on hold]

I am new to the ML and I only have the strongest will power to learn it. Question: I have the time series data and I want to approach the ML algorithm by extracting features, I didn't choose the ...
-1
votes
0answers
9 views

Product/SKU level time series forecasting [on hold]

Predict demand of product at each outlet for next 6 months 5 – 7 years of sales data at outlet level for each and every brand is available As it is a time series problem we need to design per outlet ...
0
votes
0answers
32 views

What does this proof mean?

I am self-studying Shai-Shawrts and Shai Ben-David's Understanding Machine Learning book. In chapter 4(Learning via uniform convergence) I have encountered this proof. Can someone explain me the ...
1
vote
0answers
9 views

How should I proceed to classify images of not ok microchips

So I have a new project where I have to classify different types of damages on microchips. I am new to machine learning and python in general so I am a little bit lost. I have over 100.000 images I ...