New answers tagged

0 votes

Random Forest Regressor gives negative test score in GridSearchCV

To answer your question, I consulted the following sources: sklearn.model_selection.GridSearchCV How does GridSearchCV compute training scores? sklearn.model_selection.train_test_split Firstly, it's ...
Carlos André Dos Santos Lima's user avatar
1 vote

Can any class of ML algorithms efficiently learn the modulo function (x mod y)?

The following idea can be salvaged from this paper "Grokking Modular Arithmetic" https://arxiv.org/pdf/2301.02679.pdf. We will construct a two-layer neural network which computes modular ...
dohmatob's user avatar
  • 538
2 votes

Comparison of roc-auc values ​through cross-validation for feature selection

I would suggest using penalized/regularised methods directly and avoid feature selection altogether. CV.SE has some excellent threads on the matter (e.g. Variable selection for predictive modeling ...
usεr11852's user avatar
  • 43.5k
3 votes

Comparison of roc-auc values ​through cross-validation for feature selection

The concordance probability (AUROC) is not sensitive enough for comparing models. Use a sensitive measure such as mean squared error, log-likelihood, AIC. Related information is here.
Frank Harrell's user avatar
2 votes
Accepted

Why does PyTorch Linear allow multiple output dimensions?

One use case I can think of is multivariate regression,as opposed to multvariable regression, the former being a model with more than 1 outcome variable, whereas the latter being a model with one ...
Robert Long's user avatar
0 votes

Why is ROC curve always increasing (or non-decreasing)?

Let's answer by explaining all the involved topics. What is a probability threshold? Assuming binary classification, your model should return the predicted probability of a given feature $\{x\}$ ...
Élio Pereira's user avatar
0 votes

How does whitening or decorrelation help machine learning models?

Inputing 2 (and more) correlated features (known to be multicollinear features) to the model you inflate variance of the model's output but in order to create good model you should strive to dicrease ...
JeeyCi's user avatar
  • 191
0 votes

ML Method for directional forecast

This is a binary outcome: either up or down. So many models can be used to do this, ranging from logistic regressions to random forests to neural networks. Perhaps consult the classification and ...
Dave's user avatar
  • 61k
0 votes

Why sequential forward doesn't select same feature as sequential backward?

The SBS and SFS algorithms do not give the same feature selection because they are greedy, which results in different choices depending on the "direction" (backwards of forward). The SFS ...
nchabrao's user avatar
1 vote
Accepted

Smoothness of a neural network (specifically second-order)

If we are talking about the usual fully connected / convolution layers combined with activations (i.e. not a complicated deep learning model with all sorts of layers), yes, they are second-order ...
gunes's user avatar
  • 57k
0 votes

How is the Representer theorem used in the derivation of the SVM dual form?

The representer theorem isn't needed for SVM. The solution form comes out on its own just by solving for the dual (i.e., introducing variables for the constraint, and through the KKT conditions).
MotiNK's user avatar
  • 1,499
5 votes

Why this error surface for this loss function has negative values>

However, mathematically, the loss function used by the author cannot generate negative values. Right? Correct. This loss function cannot be negative. As mentioned in comments one possibility is that ...
Robert Long's user avatar
1 vote

improving classification accuracy of the dataset as a whole by considering classifier distributions

If the problem is to estimate the relative class frequencies in a test set (or operational conditions) that may differ from those in the training set, then the best approach is to use a probabilistic ...
Dikran Marsupial's user avatar
1 vote

could someone please give an concrete example to illustrate what does Multiplicity mean in the context of Bag-of-words model?

Ignoring multiplicity (i.e., multiples instances of the same word) leads to a regular set: $$\text{go to the store to get the milk} \implies \{ \text{get}, \text{go}, \text{milk}, \text{store}, \text{...
Ben's user avatar
  • 123k
3 votes

Neural Network with 1 hidden layer with ReLU modeling capabilities

Neural networks with 1 hidden layer still have 1 output layer. So put a coefficient on each $g$ function, and then you’ll be able to fit decreasing segments.
Sycorax's user avatar
  • 90.5k
5 votes

Is F-score the same as accuracy when there are only two classes of equal size?

No. The F-score is a weighted average of precision (accuracy among predicted positives) and recall (accuracy among actual positives). Neither measure says anything at all about the group of true ...
Nuclear Hoagie's user avatar
5 votes

Is F-score the same as accuracy when there are only two classes of equal size?

NO, see the counterexample below Classify every instance the same way. You have a recall of $1$, a precision of $0.5$, and an accuracy of $0.5$. However, your $F_1$ score is: $$ F_1 = 2\dfrac{\text{...
Dave's user avatar
  • 61k
0 votes

Predict unseen samples with Graph Neural Networks

Your question seems theoretical, but I struggled to find the answer to the corresponding practical question of how to actually run inference for a single unseen node. It's well possible that many ...
user8948's user avatar
  • 302
0 votes

PCA: should standardization be applied on features or samples?

I can't figure out the reason why I should standardize with respect to the feature By no means a complete or authoritative answer... I just wanted to point out that PCA finds the axes that are ...
dariober's user avatar
  • 4,160
0 votes
Accepted

Does the Bias of the Model Only Depend on Model Class?

Consider a data generating process $$Y=f(X)+\varepsilon$$ where $\varepsilon$ is independent of $x$ with $\mathbb E(\varepsilon)=0$ and $\text{Var}(\varepsilon)=\sigma^2_\varepsilon$. According to ...
Richard Hardy's user avatar
0 votes

What is the role of temperature in Softmax?

Increasing T above 1 tends to spread the probability among the inputs, while decreasing T below 1 tends to concentrate all of the probability on the most likely class. In the limit that T reaches ...
Josiah Yoder's user avatar
0 votes

What is the best method to calculate confidence intervals on precision and recall which are not independent?

You need to account for the "repeated measurement" of a single user. You can use multiple group shuffle test/train splits where you account for the groups of data generated by a single user ...
Ggjj11's user avatar
  • 1,192
0 votes

How to define Precision when we have multiple predictions for each ground truth instance?

Clearly, there is no single correct evaluation protocol. It depends on what you aim for in your detection system. The properties of the detector that you want to quantify are typically determined by ...
Marko Lalovic's user avatar
1 vote

If the curse of dimensionality exists, how does embedding search work?

I think this question has not been answered sufficiently despite being a good question. The curse of dimensionality essentially says that random 3 vectors in sufficiently high dimensional space have ...
youneedtoread1's user avatar
6 votes
Accepted

Why is the regularization term multiplied by the error term in the cost function of SVM?

They're equivalent; just re-express $\lambda = \frac{1}{C}$ and multiply both sides by $\lambda$. Obviously, we assume $C > 0$. $$J(\mathbf{\vec w}, b) = \frac{1}{2}\|\mathbf{\vec w}\|_{2}^{2} + C \...
Sycorax's user avatar
  • 90.5k
4 votes

Linear algebra properties of a confusion matrix (eigenvalues, eigenvectors, and determinants)

The eigenvalues would really only reveal how many classes (single classifier) or how many classifiers are correlated with one another (multiple classifiers). But if you look at the quasi-diagonalized ...
Leif Peterson's user avatar
0 votes

How to handle correlated variables before using Recursive Feature Elimination?

Tree classifiers typically use features independently (univariate) for node splits and are not really multivariate. Thus, they're not likely to knock down the importance score of a feature because it'...
Leif Peterson's user avatar
0 votes

Categorical Intra-Cluster Quality - Suggestions

Based on a distance measure (e.g., simple matching distance) you can compute the silhouettes and plot them (R-command silhouette). You can also have a look at ...
Christian Hennig's user avatar
0 votes

K means clustering of image with k=1 vs mean of all pixels

Welcome to CrossValidated! If when you say "k-means 1" you mean a k-mean algorithm with only 1 cluster, then I think the centers of the cluster are represented by the multivariate mean ...
jmarkov's user avatar
  • 683
8 votes
Accepted

Is it really so bad to do SMOTE on the training set before crossvalidation?

To start, let's point out that using SMOTE on the entire dataset before a test set is split off is bad: as you say, it causes data leakage, and thus generally overly-optimistic estimates of ...
Ben Reiniger's user avatar
  • 4,409
7 votes

Is it really so bad to do SMOTE on the training set before crossvalidation?

First of all, you need to know that the general consensus is turning against SMOTE. It doesn't really solve the problem it is supposed to solve, worse, the actual problem doesn't really exists. ...
Lucas Morin's user avatar
  • 1,505
4 votes
Accepted

How to incorporate prior knowledge after ML training?

You can either use an ensemble of multiple classifiers that will improve performance when your ML classifier breaks down[1], or implement boosting for your ML classifier assuming that it's a "...
Leif Peterson's user avatar
1 vote

Producing samples from exponential family conditional on minimal sufficient statistic

This paper looks exactly at this question? Use of the Gibbs Sampler to Obtain Conditional Tests, with Applications by Lockhart, O'Reilly and Stephens https://academic.oup.com/biomet/article-abstract/...
KAnayaI's user avatar
  • 11
0 votes

The Impact of Vector Magnitudes in Recommendation Systems Matrix Factorization Models

There actually has been some research done on this problem. Here's one example, which focuses on sparse vectors and on the computational difficulties associated with constraining vector magnitudes.
Bill Vander Lugt's user avatar
4 votes
Accepted

Can a ML classifier's prediction be understood as a probability?

That would be desirable, but it is not guaranteed to make as much sense as we might like. First, you could make an argument that any predicted $p(\mathcal C_k|\mathbf x_i)\in[0,1]$ is a probability in ...
Dave's user avatar
  • 61k
2 votes
Accepted

LSTM input vector

The $X_t$ are the measurements at time $t$. If you have multiple measurements at each $t$ (for instance, temperature at time $t$ and humidity at time $t$), then $X_t$ is the vector holding all of the ...
Sycorax's user avatar
  • 90.5k
1 vote
Accepted

Time series model selection based on demand

Your question is rather broad. I will give a few thoughts off the top of my head. First off, I recommend this forecasting workflow. (Little surprise there, since I wrote it up.) Decide what kind of ...
Stephan Kolassa's user avatar
1 vote

Comparing probability threshold graphs for F1 score for different models

The three models are likely almost the same, just with a monotonic shift of their predicted probabilities (see King and Zeng 2001, and DS.SE "Is There a Way to Re-Calibrate Predicted ...
Ben Reiniger's user avatar
  • 4,409
3 votes

Is it okay to use different data preparation procedures for different models?

I wonder if, for your specific example, this counts as "data preparation" at all. While it's true that collinearity is a feature only of the independent variables, the usual procedure for ...
Peter Flom's user avatar
  • 117k
4 votes
Accepted

Is it okay to use different data preparation procedures for different models?

Selecting (or constructing) variables is very much dependent on the model, e.g. if trying to model a quadratic relationship with a linear model you need to include an additional variable to capture ...
user2974951's user avatar
  • 7,753
0 votes

What's the difference between variance scaling initializer and xavier initializer?

To understand the difference of each initialization, we need to undersetand what is going on inside the neural network (NN) forward and backward propagation and how to manage the neuron output signals ...
mon's user avatar
  • 1,448
1 vote

Find marginal distribution of $K$-variate Dirichlet

It is possible to derive this integral using the expression for the normalization constant of the dirichlet distribution. The key idea is to recast the integral containing the terms not involving $x_j$...
Gaurav Dhir's user avatar
0 votes

Why do we work with factor of likelihoods instead of e.g. a sum for a batch in the negative log likelihood loss function?

The "likelihood" is a derivative concept. The primary concept is probabilistic: we consider the joint density of a sample of independent observations, because it expresses the probability of ...
Alecos Papadopoulos's user avatar
3 votes

Why do we work with factor of likelihoods instead of e.g. a sum for a batch in the negative log likelihood loss function?

Commonly, in statistics, we make a strong distributional assumptions about where each datapoint comes from: we say that $P(y_i|\theta) = f(y_i,\theta)$. If we make the assumption that the data are ...
John Madden's user avatar
  • 4,085

Top 50 recent answers are included