Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
1
vote
1answer
6 views

Why is that in reinforcement learning, the policy is a function of the state (but not the or just the reward?)

In reinforcement learning, we seek an optimal policy which is defined as a mapping $\pi: s \mapsto a$ from the set of states to the set of actions, or a mapping $\pi: s \mapsto p(a|s)$, $p(a|s)$ is a ...
1
vote
0answers
9 views

Why Does My Model Perform Better Without Dimensionality Reduction (features > samples)

I created a classifier (a linear SVM in scikit-learn) to classify tweets about the fat acceptance movement (yeah that's a thing) as supporting the movement, opposing the movement, or having an unclear ...
1
vote
1answer
9 views

Does mislabeling due to adversarial noise in features count as adversarial machine learning?

According to the traditional definition, Adversarial machine learning is a technique employed in the field of machine learning which attempts to fool models through malicious input. However, I have ...
3
votes
1answer
12 views

How do I sample a multivariate posterior when I can sample the likelihood and prior?

Suppose I want to sample the posterior distribution of a multivariate $\beta$ given some scalar $x$. By Bayes' theorem, this distribution is $$P(\beta|x) \propto P(x|\beta)P(\beta) $$ I don't have ...
1
vote
0answers
13 views

Neural Network - Estimating Non-linear function

I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, ...
3
votes
0answers
19 views

Necessary conditions for Independence in multivariate Gaussian

Let $\mathbf{y}_1, \mathbf{y}_2, \mathbf{y}_3$ be random vectors belonging to a joint multivariate Gaussian distribution with mean $\mathbf{0}$ and covariance matrix $\mathbf{C}$ . What are the ...
0
votes
1answer
11 views

Good source for learning about Binary Classification

When dealing with binary classification, I most often find myself estimating a logistic regression model. I have tried a few other approaches as well, but to be honest I feel like I know way too ...
0
votes
1answer
336 views

Using cosine similarity to measure similarity between uses is not correct

I have a theoretical question. I have implemented a recommender system using collaborative filtering method. There, I am using cosine similarity method to calculate similarity between two users. I ...
0
votes
1answer
10 views

How to interpret recommender scores in implicit matrix factorization?

I am using Alternating Least Squares model from the Implicit library on the LastFM dataset, recommending artists to various users. The input data is simply a sparse matrix of users, artists, number ...
1
vote
0answers
11 views

How does high dimensional data residing in low dimensional manifold have high variance?

I was pondering over the fact that for a very high dimensional data, if it lies in a low dimesional manifold, then in the bias variance decomposition for it, the variance will be high and bias will be ...
0
votes
1answer
429 views

On the convergence of Iterated Conditional Modes (ICM) for MAP inference

ICM is very fast but I could not find any references that contain a detailed analysis on its convergence (e.g. rate of convergence). Any suggestions please? Thanks a lot for your help!
1
vote
1answer
28 views

In supervised learning, why maximize the joint likelihood when we are interested in maximizing individual likelihood?

Suppose I have a data set $\{(x^i, t^i)\}_{i =1, \ldots, n}$ generated i.i.d. $t^{i} \in \{1, -1\}$ are binary targets. We would like to run the logistic regression, which is based on maximizing the ...
1
vote
0answers
6 views

Deriving the un-normalized log posterior on sigma

Hello I'm trying to derive the un-normalized log posterior on sigma this is what I have so far $$ p \left(\sigma \mid \mathbf{x}, \mu\right) \propto \log \left( p \left(\sigma \mid \mathbf{x}, \mu\...
1
vote
0answers
14 views

Can solving Linear Regression result into zero coefficient of any variable?

I was reading about ridge and lasso regularization and I have a question that if solving regression might result in zero coefficient of any variable if it is not useful in prediction at all. For ...
0
votes
0answers
50 views

What happens if I use categorical cross-entropy for one class detection

I have used categorical cross-entropy instead of binary cross-entropy for one class detection. If the results are wrong, does the proportion maintain? Like, the worst result that I got is still the ...
2
votes
3answers
64 views

Bishop PRML Question 8.10: d-separation

I have trouble with solving the second part of question 8.10 from Bishop's PRML (attached as image). I tried several things. Here's my latest attempt: \begin{align} p(a, b, d) &= \int p(a)p(b)p(...
0
votes
0answers
13 views

What is 'wresid' in statsmodels library?

To obtain Statmodel summary parameter, I need 'wresid'. I am looking for its explanation and how it helps to calculate other information? What is the meaning of 'transformed/whitened regressand and ...
0
votes
1answer
117 views

Q-learning agent stucks in an infinite loop

I am simulating a mouse to find a cheese on an empty table. I randomly put a cheese on the table and let the mouse find the cheese without falling off the table. The problem is, in test part, agent ...
2
votes
0answers
38 views

Tests for significant difference between prediction errors of machine learning regression models

Are there any recommended statistical tests for significant differences between cross-validation (CV) estimates of prediction error of machine learning regression models? I have 5 different random ...
2
votes
1answer
163 views

Can this network learn the XOR function?

Let's say I have the following constraints: The architecture is fixed (see image) (note that there are no biases) Activation function for the hidden layer is ReLU There's no activation function for ...
0
votes
0answers
22 views

How can I tell if I am overtraining my support vector machine?

I am trying to train a Support Vector Machine (SVM) classifier to classify various items into 5 categories. I have trained two SVM classifiers, however, I am concerned that the accuracies and F1 ...
0
votes
2answers
23 views

Why is the equation for a single-neuron perceptron decision boundary Wp + b = 0 set to ZERO?

I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. ...
0
votes
2answers
4k views

Conceptual limits on mtry in Random Forest algorithm

I have a random forest being applied to 7 different input variables to predict a particular classification. I've done a grid search on the hyperparameters mtry and ...
0
votes
1answer
17 views

how to interpret predict.proba function in xgboost?

I am using xgboost to predict probability for binary classification and rank the probability distribution to create rank order classification. I want to know what the score means? does it represent ...
0
votes
0answers
10 views

How do Variational Autoencoders use Negative Log Likelihood/Cross entropy on real valued outputs?

When training a Variational Autoencoder, the function being maximised is the expected lower bound: $$ \mathscr{L}(\boldsymbol{\theta}, \phi; \mathbf{x}^{(i)}) = -D_{KL}\left(q_{\phi}(\mathbf{z}|\...
0
votes
1answer
32 views

Parameter Estimation via MCMC

In general, we use MCMC method to sample from a distribution which is hard to compute. In Bayesian setting, we sample from the posterior distribution of the random parameters defining the underlying ...
3
votes
2answers
296 views

Bayes optimal decision for logistic regression: Self-study exercise

We want to find the Bayes optimal decision for logistic regression. That means that the goal is to find the actions, which minimize our expected loss (also often called expected cost or risk). Here ...
0
votes
0answers
22 views

Comparison of ML models

There are two models f, g trained on some labelled (x,y), where y has 2 classes. During testing they correctly predict the same unseen samples. However, the probability they output are different. So ...
107
votes
4answers
66k views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
0
votes
1answer
16 views

Cross validation and synthectic data-sets

In an machine learning context, does it make sense to carry out cross validation if the data is generated synthetically? To test for generalization, isn't it better in this context to simply ...
48
votes
3answers
20k views

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by ...
0
votes
0answers
21 views

Generate more data for a small dataset

I have been working on a dataset which has 14 attributes and 303 rows(instances) along with the binary labels. I want to generate more data so that I could train my neural networks so that I could ...
5
votes
2answers
219 views

Is the optimization of the Gaussian VAE well-posed?

In a Variational Autoencoder (VAE), given some data $x$ and latent variables $t$ with prior distribution $p(t) = \mathcal{N}(t \mid 0, I)$, the encoder aims to learn a distribution $q_{\phi}(t)$ that ...
1
vote
1answer
62 views

Does adding new categorical data decrease prediction performance in classification?

I have a dataset in which new data comes in everyday. There are categorical variables in the inputs. As a result, I use one-hot-encoder to create a dummy variable. If a new categorical comes in, the ...
6
votes
1answer
656 views

Multi-label classification: Predict product category

I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to ...
0
votes
0answers
12 views

Check when dataset should be splitted [closed]

I have a dataset where I need to say if it could be divided into two or more to help the AI to classify that dataframe. Therefore, I applied the elbow method and the silhouette to see into how many ...
2
votes
0answers
20 views

Python Machine Learning: Measuring confidence in individual classifications

I am pretty new to machine learning. I am currently using sci-kit learn's DecisionTreeClassifier and RandomForestClassifier to look at astronomical data. It takes in a number of parameters and tries ...
2
votes
1answer
46 views
+50

Implication of marginal independence of features for classification

This question is a follow-up to my earlier question on naive Bayes (NB) classification. The example we're considering is that of spam classification, in which an email is classified as spam ($S \in \{...
15
votes
3answers
10k views

Interpreting negative cosine similarity

My question may be a silly one. So I shall apologize in advance. I was trying to use the GLOVE model pre-trained by Stanford NLP group (link). However, I noticed that my similarity results showed ...
0
votes
1answer
18 views

Derivations of Forward and Reverse KL Divergence equations

In the Forward KL, the entropy has disappeared and in the Reverse KL, the entropy has a plus sign, why are they so?
0
votes
0answers
19 views

Differences between the lm(), lqs(), and rq() function

In R, we can use the build-in function lm() for linear regresson. However, we also use the lqs() function from ...
2
votes
2answers
72 views

(Linear regression) Can I train and validate at the same time using the following approach?

In a lot of material I found online, training and validation seems to be an iterative process For example, the regularized regression problem $E = \|Xw - t\|_2^2 + \lambda \|w\|^2_2$ $X$ is data ...
0
votes
2answers
49 views

How important is p-value in Machine Learning?

Scikit-Learn doesn't exhibit the p-values for your models. I'm used to look at the p-values - besides a few other factors - when choosing the variables to consider on my final model. However, p-values ...
2
votes
1answer
231 views

classification on imbalanced dataset via random forest: results vary with random seed

I have a highly imbalanced dataset of about 8000 observations, with 11 features and one binary target variable. I want to predict the target labels, considering that the "1" target label occurs for 1....
3
votes
2answers
1k views

Why use different cost function for linear and logistic regression?

I mean least squares already penalize one big mistake more, then several small ones. So why don't just leave same "mean square error" for logistic regression - it is simpler than messy formula with ...
0
votes
1answer
116 views

Neural Network border condition or extrapolation data

I was reading the user guide of NN for Matlab and I found this quote about extrapolation data: It is important that the data cover the range of inputs for which the network will be used. ...
2
votes
1answer
115 views

How does one generate (smooth) varying size output signals with Machine Learning?

I am interested in knowing about generative methods that generate signals (e.g. images) of varying sizes. But the size generation being sort of "smooth/continuous". So for example, generating images ...
6
votes
1answer
565 views

Machine learning algorithms as matrix factorization

I came to know that various ML algorithms can be posed a matrix factorization problems with different constraints specific to that particular problem. Is there any good material that provides an ...
4
votes
1answer
61 views

Topological rather than metric based machine learning theory?

The first notion of continuity in a math class is usually the one based on metric spaces. In particular, the $\epsilon,\delta$ definition of continuity. But in topology, a more general notion of ...
0
votes
0answers
9 views

What is in LSTM memory cell

I understand the structure of LSTM, but I am unclear what is in the vector holding the LSTM memory, and I have not found any reference providing intuitive example on what exactly that vector is ...