New answers tagged machine-learning
1
vote
What distribution assumptions do Gupta, Podkopaev & Ramdas (NeuroIPS 2020) think could be made?
As far as I can tell, in a binary “classification” model, the outcome is conditionally Bernoulli, end of discussion. Thus, it does not seem like there are any distributional assumptions to make.
The ...
0
votes
Including Collider Variables in Prediction
Agree with @JulianSchuessler. You should consider causal inference if you will make inference (forgive the redundancy) on the learnt model (i.e. the variable associations). But for mere black box ...
4
votes
Accepted
Two-sided KS-Test for Evaluating Prediction Model?
EDIT
I earlier claimed that the Kolmogorov-Smirnov statistic yields a proper scoring rule. That does not appear to be the case, judging from Thorarinsdottir, 2012, "Proper scoring rules and ...
1
vote
Scaling laws for neural network memorization
The best paper I have seen on the subject of scaling laws for neural network memorization was published at ICLR last year: Provable Memorization Capacity of Transformers. Since you "don't care ...
0
votes
Compare two datasets and whether they agree
You do not need to give up as Kamil Kaczmarek suggests, and you also do not need to shoehorn a transformed variable into a logistic regression as Peter Flom suggested (although to be fair to his ...
2
votes
Accepted
How to analyze uncorrelated data?
In some sense, you already have analyzed the data. After all, you determined that that there is minimal (Pearson) correlation between the target and the proposed features. That sure seems like ...
0
votes
Problems Determining Optimal Cluster Number for Time-Series Data
You can use results from K-means to explore cluster validity using other methods, based on different distance methods. Firstly, I would recommend looking at the effect of various feature ...
0
votes
Compare two datasets and whether they agree
I am not so sanguine about assuming 1 to 3 = bad and 4 to 5 = good. But, if you do that, then you can do logistic regression with "marital status" as the dependent variable and "group&...
0
votes
Does a model learn the same attention scores when retrained?
The attention scores of a model directly depend on the weights of the model.
Usually re-training a model (unless fixing the seed) will not lead to the same model weights, and for that reason, in many ...
1
vote
Features available during training but not at prediction
Your question relates to a few different and loosely-related concepts, so I'll try to address each of them.
(1)
In short, SHAP value of a feature $x_j$ is the average absolute contribution of this ...
2
votes
Binary decision boundary requiring 2 hidden layers in neural network with limited neurons
Neural networks with one hidden layer are universal approximators (given some constraints), so you won't find such a class of functions. See Does the universal approximation theorem apply to ReLu?
1
vote
Scaling laws for neural network memorization
I've thought about this some more and I'm not sure if I can get you an exact "minimum number" or a tight scaling law, but I can get you some bounds if we make many simplifying assumptions.
...
0
votes
How to evaluate multi-class classifier on probability prediction task?
What I suspect you have read is that classification accuracy becomes problematic in the case of imbalance. There are valid arguments that classification accuracy is problematic in the balanced setting,...
0
votes
Accepted
Calculating the True Error of a Distribution
You're missing chain rule step which is $D((a,0))=D(a)D(0|a)=\frac{1}{2}(\frac{1}{2} - \epsilon), D((b,1))=D(b)D(1|b)=\frac{1}{2}(\frac{1}{2} - \epsilon)$
So after straightforward substituting your ...
1
vote
Find event date given the probabilities of finding an event
If you are starting with probabilities of event, then I don't think you can find the event date. You can find the probability of event taking place on a specific date.
You could model this as Poisson ...
0
votes
Literature on discriminatory model VS discriminatory feature
Question is not totally clear, and you would benefit from giving more details. But try logistic regression, even if individual predictor does not discriminate between the two groups, there might be ...
0
votes
What is the relationship between estimation error, approximation error, bias, variance in machine learning?
The book, and the post above, are not quite correct.
In general, bias is not approximation error, and variance is not estimation error. It is not even the case that one is a special case of the other,...
3
votes
Is it really so bad to do SMOTE on the training set before crossvalidation?
I understand that doing this leads to data leakage, but if I get better performance on the test set does it really matter?
YES, it matters! The problem with data leakage is that you can no longer ...
4
votes
Is it really so bad to do SMOTE on the training set before crossvalidation?
Yes, it is bad, and it isn't just leakage (+1 to the previous answers). This is because while the synthetic data generated by SMOTE is asymptotically from the same distribution as the data, this is ...
0
votes
Training with a target value as feature
The only thing that limits which features you should use for prediction are which features you actually expect to have when making a prediction. If there are scenarios where you will know y-hat and x ...
1
vote
Accepted
Maximum Likelihood - Information Matrix Identity Derivation
This is lot easier to show if you use the fact that the expectation of the score is always zero, so
$$
\mathbb{E}(s(\lambda;\mathbf{X})^2)=\mathrm{Var}(s(\lambda;\mathbf{X}))
$$
You then have
$$
\...
1
vote
can i have multiple similar features derived from same properties in a dataset?
It depends on the nature of your data.
If you have a massive sample size and your variables have low noise, adding more variables will make your model better.
If you have a tiny sample size and your ...
2
votes
Accepted
robust linear regression with interaction
Let's run through the model parameters and emmeans. Your model itself will predict the mean epigenetic age given a set of predictor variables. I'm not so sure a ...
1
vote
Can any class of ML algorithms efficiently learn the modulo function (x mod y)?
i would say the answer is 'no', you need a different representation to generalise the mod function as opposed to just memorising the mod for a fixed range of numbers.
simple solution (make input the ...
1
vote
simple ANN as a set of linear transformations
I think your specific problem is:
However, we can achieve this by using two perceptrons in the hidden layer and one for the output layer, without using an activation function
If you write out the ...
0
votes
simple ANN as a set of linear transformations
You cannot achieve this using multiple layers and more perceptrons without non-linear activation functions. As you noticed, if no activation is used, any dense network, no matter how many layers and ...
2
votes
Which Forecast Evaluation Metric To Use?
The evaluation function and the scaling are distinct issues in my mind. To me, scaling to $0$-$100$ is straightforward: compare to a reasonable baseline model. This is what the usual $R^2$ does by ...
0
votes
Random Forest Regressor gives negative test score in GridSearchCV
To answer your question, I consulted the following sources:
sklearn.model_selection.GridSearchCV
How does GridSearchCV compute training scores?
sklearn.model_selection.train_test_split
Firstly, it's ...
2
votes
Can any class of ML algorithms efficiently learn the modulo function (x mod y)?
The following idea can be salvaged from this paper "Grokking Modular Arithmetic" https://arxiv.org/pdf/2301.02679.pdf. We will construct a two-layer neural network which computes modular ...
5
votes
Comparison of roc-auc values through cross-validation for feature selection
I would suggest using penalized/regularised methods directly and avoid feature selection altogether.
CV.SE has some excellent threads on the matter (e.g. Variable selection for predictive modeling ...
6
votes
Comparison of roc-auc values through cross-validation for feature selection
The concordance probability (AUROC) is not sensitive enough for comparing models. Use a sensitive measure such as mean squared error, log-likelihood, AIC. Related information is here.
3
votes
Accepted
Why does PyTorch Linear allow multiple output dimensions?
One use case I can think of is multivariate regression,as opposed to multvariable regression, the former being a model with more than 1 outcome variable, whereas the latter being a model with one ...
0
votes
Why is ROC curve always increasing (or non-decreasing)?
Let's answer by explaining all the involved topics.
What is a probability threshold?
Assuming binary classification, your model should return the predicted probability of a given feature $\{x\}$ ...
0
votes
How does whitening or decorrelation help machine learning models?
Inputing 2 (and more) correlated features (known to be multicollinear features) to the model you inflate variance of the model's output but in order to create good model you should strive to dicrease ...
0
votes
ML Method for directional forecast
This is a binary outcome: either up or down. So many models can be used to do this, ranging from logistic regressions to random forests to neural networks. Perhaps consult the classification and ...
0
votes
Why sequential forward doesn't select same feature as sequential backward?
The SBS and SFS algorithms do not give the same feature selection because they are greedy, which results in different choices depending on the "direction" (backwards of forward).
The SFS ...
2
votes
Accepted
Smoothness of a neural network (specifically second-order)
If we are talking about the usual fully connected / convolution layers combined with activations (i.e. not a complicated deep learning model with all sorts of layers), yes, they are second-order ...
0
votes
How is the Representer theorem used in the derivation of the SVM dual form?
The representer theorem isn't needed for SVM. The solution form comes out on its own just by solving for the dual (i.e., introducing variables for the constraint, and through the KKT conditions).
6
votes
Why this error surface for this loss function has negative values>
However, mathematically, the loss function used by the author cannot generate negative values. Right?
Correct. This loss function cannot be negative.
As mentioned in comments one possibility is that ...
2
votes
improving classification accuracy of the dataset as a whole by considering classifier distributions
If the problem is to estimate the relative class frequencies in a test set (or operational conditions) that may differ from those in the training set, then the best approach is to use a probabilistic ...
1
vote
could someone please give an concrete example to illustrate what does Multiplicity mean in the context of Bag-of-words model?
Ignoring multiplicity (i.e., multiples instances of the same word) leads to a regular set:
$$\text{go to the store to get the milk} \implies
\{ \text{get}, \text{go}, \text{milk}, \text{store}, \text{...
4
votes
Neural Network with 1 hidden layer with ReLU modeling capabilities
Neural networks with 1 hidden layer still have 1 output layer. So put a coefficient on each $g$ function, and then you’ll be able to fit decreasing segments.
6
votes
Is F-score the same as accuracy when there are only two classes of equal size?
No.
The F-score is a weighted average of precision (accuracy among predicted positives) and recall (accuracy among actual positives). Neither measure says anything at all about the group of true ...
5
votes
Is F-score the same as accuracy when there are only two classes of equal size?
NO, see the counterexample below
Classify every instance the same way. You have a recall of $1$, a precision of $0.5$, and an accuracy of $0.5$. However, your $F_1$ score is:
$$
F_1 = 2\dfrac{\text{...
0
votes
Predict unseen samples with Graph Neural Networks
Your question seems theoretical, but I struggled to find the answer to the corresponding practical question of how to actually run inference for a single unseen node. It's well possible that many ...
0
votes
PCA: should standardization be applied on features or samples?
I can't figure out the reason why I should standardize with respect to the feature
By no means a complete or authoritative answer... I just wanted to point out that PCA finds the axes that are ...
0
votes
Accepted
Does the Bias of the Model Only Depend on Model Class?
Consider a data generating process
$$Y=f(X)+\varepsilon$$
where $\varepsilon$ is independent of $x$ with $\mathbb E(\varepsilon)=0$ and $\text{Var}(\varepsilon)=\sigma^2_\varepsilon$. According to ...
0
votes
What is the role of temperature in Softmax?
Increasing T above 1 tends to spread the probability among the inputs, while decreasing T below 1 tends to concentrate all of the probability on the most likely class.
In the limit that T reaches ...
0
votes
What is the best method to calculate confidence intervals on precision and recall which are not independent?
You need to account for the "repeated measurement" of a single user.
You can use multiple group shuffle test/train splits where you account for the groups of data generated by a single user
...
Top 50 recent answers are included
Related Tags
machine-learning × 20232neural-networks × 4050
classification × 2680
regression × 2243
r × 1222
python × 1094
svm × 1045
cross-validation × 1000
mathematical-statistics × 993
predictive-models × 950
time-series × 930
random-forest × 856
optimization × 665
feature-selection × 662
scikit-learn × 627
logistic × 615
clustering × 600
boosting × 577
probability × 571
conv-neural-network × 535
bayesian × 505
data-mining × 495
natural-language × 412
supervised-learning × 387
cart × 380