Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
1
vote
0answers
6 views

What is the effect of changing the weight decay and warm-up steps in fine-tuning PEGASUS?

I am fine-tuning PEGASUS model using this script. I am currently using the SAMSum dataset and I have reached a point in which the output doesn't get better. Examples: The Actual Summary Alexis and ...
0
votes
0answers
23 views

how to explain accuracy score of 0.51 when training and test scores are around 0.79? [closed]

I am trying to solve some machine learning problem, I dont understand how accuracy can be so lower than training score and test score
0
votes
0answers
15 views

Can Mutual Information based feature selection be used when the input variables are numerical and the output is categorical?

I am working on a machine learning project for a classification problem. In the dataset the input variables are numerical and the output is categorical. Is it appropriate to apply the Mutual ...
0
votes
1answer
12 views

Is it possible higher validation accuracy but lower test accuracy?

Let's suppose we train a model with 10-fold cross validation. For hyperparameter selection, one can take all combinations of hyperparameters using grid-search. My question: can the test accuracy be ...
0
votes
0answers
10 views

Underfitting - how to tell if model capacity is saturated or if model is difficult to train?

I have a dense CNN which is very slow to train and additionally exhibits unusual loss behaviour. This is illustrated by the graphs below: for the graphs with two curves blue is the training curve and ...
0
votes
0answers
10 views

NLP - train a model

I have a large set of large texts (around 60K texts, each one having between 100 and 30K ...
0
votes
0answers
16 views

Normalizing Flows Applied to Dynamical Systems

Suppose $x : [t_0,t_f] \rightarrow \mathbb{R}^{n}$ is a trajectory that satisfies the linear ODE $$ \dot{x} = Ax, \quad x_0 \sim \rho_0, $$ where $\rho_0$ is the PDF of the initial state $x_0$. Thus, ...
1
vote
1answer
18 views

Does the transformers model (in “Attention is All You Need”) exclude the encoder in language modelling tasks?

The language model I am referring to is the one outlined in "Attention is All You Need": My understanding is (please correct me if I am wrong) that when the task is translation, the encoder'...
1
vote
0answers
12 views

Wrong application of cross validation. Is nested cross validation better? [closed]

Assume we want to use k-fold cross-validation to get an estimate for the expected prediction error. Let's assume we use SVM with the hyperparameters fixed. From my understanding, in each iteration of ...
0
votes
1answer
20 views

Confidence interval for Regression Trees

Many people think the regression tree is only an algorithm and it doesn't make sense approach confidence interval to it so I'd like to know if there's anyone figured out how to do it. A regression ...
0
votes
0answers
8 views

Focus on predicting top n values

Assume a dataset of size $(m,p)$ where $m$ is the number of rows and $p$ is the number of columns. Let, $y : (m,1)$ be the continuous (regression) variable I need to predict. But my final use case is ...
1
vote
1answer
18 views

Calculate AUC based on TPR and FPR

I have two equations for TPR and FPR (based on the threshold t), for example: TPR = (1-t)^2 FPR = (1-t)^0.2 How can I calculate ...
0
votes
1answer
7 views

Greedy policy definition

I've always seen as definition for the greedy policy the one that maximizes the action value function $q_{\pi} (s,a)$ over the actions $a$. How is this equivalent to the following one that I found on ...
1
vote
1answer
29 views

Bootstrapping the pipeline itself

Generally, I've been taught that the test set should not be touched by any means, but suppose I have a pipeline that: First split data into ...
0
votes
1answer
33 views

Can we find upper bound for loss functions?

Is it easy to find upper bound for loss functions like 0-1 loss and hinge loss ?!. I always find this sentence, which is "hinge loss is an upper bound of 0-1 loss", Can we compute the upper ...
0
votes
0answers
10 views

Average vs Max Pooling in Modified DenseNet Transformation Blocks

A similar question has already been asked and answered in regards to why DenseNet uses average pooling instead of max pooling in DenseNet, and the answer seems satisfactory to me. However I'm using a ...
1
vote
0answers
14 views

How can I use RNN to predict 5 sequence numbers based on historical data?

Supposing I want to predict the next 5 sequences of numbers based on the historical data. my training set is as follow: historical data: 3634 38 51 190 127 2422 568 5796 578 60 1935 -> next_value: ...
1
vote
1answer
19 views

paired t test to check if features in test and holdout datasets are from same distribution

The machine learning model we are developing is having different resuls on test and holdout and we want to check if the test and holdout are from the same distributions or if there is a feature drift ...
0
votes
0answers
10 views

What are some typical use-cases for machine learning models with “moderate” predictive performance

For example, say you have a classifier with an AUC of 0.75 (not good but not bad). Are there any typical uses cases where predictive performance does not have to be very good?
0
votes
0answers
41 views

How can Hinge loss be upper bound of the 0-1 loss?

I just have a question and I did not find clear explanation, hinge loss is upper bound of 0-1 loss, so does this mean upper bound regard to a single point loss such as 0 or 1 ?!. So, for example the 0-...
1
vote
0answers
13 views

Rolling forecast vs. static training data for financial timeseries?

I want to train a statistical model to predict financial asset returns. I'm wondering whether it would be more effective to train a rolling forecast model rather than training a single model with a ...
2
votes
1answer
15 views

xgboost demand model with a smooth effect for the price variable

Question The question is: how to smooth out kinks in individual demand curves in a GBDT model without underfitting on the price variable? Background We have some GBDTs demand models already in place (...
1
vote
0answers
13 views

What does one imply by the term “overgeneralization” in machine learning?

I know overfitting and underfitting in machine learning context, and what generalisation means as well. But, recently I was introduced to an uncommon terminology "overgeneralization" in ...
0
votes
0answers
9 views

Gradient Boosting vs XGBoost key differences

I know XGBoost minimize a regularized loss function instead GB (gradient boosting) but I dont know how trees grow, it would be a simple fit to estimate G/H? where G is first derivate with respect to ...
0
votes
0answers
15 views

Why can Ensemble learning cause overfitting?

When I use some model that uses ensemble learning (for example random forests, AdaBoost, XGBoost) there is a limit to the number of estimators (after a certain number, the accuracy of the model begins ...
0
votes
0answers
29 views

How to do Maximum Likelihood Estimation (MLE) of a Poisson Regression using numpy

I am currently trying to learn how MLE in a poisson regression context works. As such I am trying to compute a poisson regression from scratch using numpy. Furthermore, I try to solve the MLE using ...
1
vote
0answers
18 views

Data Generating Process Simulation for Entity Resolution

Entity resolution across multiple databases with millions of entities seems to be quite a laborious learning task. Right now, I am combining a variety of methods to come up with confident estimates. ...
0
votes
0answers
3 views

When to use Transfer learning and fine Tuning in machine learning?

Can anyone explain some use cases when to use transfer learning and Fine-Tuning in machine learning ? I am always have confusion on it .
1
vote
0answers
13 views

Rate of convergence of Machine learning models

I am currently doing some work on the double debiased machine learning algorithm by Chernozhukov et al. 2016. They achieve $ \sqrt{N} $ rate of convergence for estimation of a treatment parameter. ...
2
votes
1answer
63 views

Is $-\sum_{n=1}^{N} \log(1+\exp(-t_ny_n)) $ the same loss as $\sum_{n=1}^{N}\{ t_n\log(y_n) + (1 - t_n)\log (1-y_n) \}$?

I am trying to understand different forms of loss functions. I get confused with the terms cross entropy-loss and negative log-likelihood losses. I have seen the two following definitions of cross ...
1
vote
0answers
16 views

What is the correct way to calculate the covariate-specific effect in causal inference?

My question is related to the concept named "Covariate-Specific Effects" in the book "Causal Inference in Statistics: A Primer". In Section 3.3, it is called the "w-specific ...
0
votes
0answers
11 views

Perceptron learning rule - Recover input patterns

Doing exercises in my Deep Learning book I'm stuck on this following exercise as the correct answer is marked without any explaination: Using classical perceptron learning rule starting with W0 = [2.4 ...
0
votes
0answers
31 views

Can I tell my model is overfittng?

I am developing fully convolutional model for semantic segmentation task and I tried to use spatial dropout layers to prevent overfitting of my model. My model has interesting learning curves and I am ...
0
votes
0answers
16 views

Question about some recipes from Andrej Karpathy blog [closed]

Recently I found for me an interesting and, in my opinion, useful Andrej Karpathy blog (http://karpathy.github.io/2019/04/25/recipe/) about training neural networks. Since I am new in neural networks, ...
0
votes
1answer
15 views

When do we require to calculate the confidence Interval?

I am using various machine learning algorithms for last 7 years. To validate the model in classification algorithm we use precision, recall, f1 score. For regression methods we use R^2,RMSE kind of ...
0
votes
0answers
113 views
+50

Do random forests work better than multinomial logistic regression for prediction of categorical non-binary variables? Why?

I posted another question that was well received. I am posting this new question because it was suggested by other members of Cross Validated. Here is the link of the original question that I posted: ...
0
votes
1answer
36 views

Good reference book for linear statistical models? [duplicate]

My Ph.D. training is mostly in applied mathematics. I'm interested in learning materials of linear statistical models, especially their applications in data science or statistical machine learning. Is ...
0
votes
0answers
12 views

Specifying parameter grid for regression model

I am working with more than one dataset. So, I have to test my Random Forest Model over 4 datasets. The parameter grid I am taking for dataset D1 is not producing good results for dataset D2 and so on....
0
votes
0answers
12 views

How can I use deep learning to reduce false negatives in NLP specific task like cyberbullying classification? [closed]

I would like to experiment whether deep learning approaches can reduce the false negatives count than traditional machine learning approaches in Cyberbullying classification task. Can anyone help me ...
0
votes
0answers
18 views

Max Pooling vs Average Pooling for residual/skip connections

I've implemented a CNN with skip connections; some connections skip across residual blocks with no spatial downsampling but some connections skip across blocks that have convolutions with a stride of ...
1
vote
0answers
15 views

L1 loss giving a better result than L2 loss for optimizing PSNR in an image super resolution problem

in an Image Super Resolution kind of problem, I want to get the highest PSNR values for the super resolution images from the low resolution images obtained after training a model. I experimented with ...
-1
votes
0answers
22 views

Curse of dimensionality and model overfitting/underfitting, which one is appropriate? [closed]

During a technical interview in a high-tech company, they asked me the following question: Does training with a high number of features with a relatively low number of samples (i.e. rectangular ...
0
votes
0answers
25 views

Why is forward Kullback Leibler Divergence mean seeking?

I have a course on Information theory, in the which we talk about forward KLD in order to approximate pdfs. There is an example that's the same example as on this blog : https://towardsdatascience.com/...
2
votes
0answers
26 views

Kernel trick to logistic regression

Why can't I apply the kernel trick in logistic regression? My reasoning is: in SVM the logit is: $z = \sum_i \alpha_i K(x_i, x) + b$ Where K is the kernel function. In logistic regression you have ...
0
votes
0answers
9 views

When training data is much less than the prediction perdio

Given training data on tweets and their retweets, how would you predict the number of retweets of a given tweet after 7 days after only observing 2 days worth of data? Its strikes me that this ...
0
votes
0answers
10 views

Feature selection before hyperparameter selection & cross-validation

I'm trying to train a model to predict water solubility and the dataset has 200 features, with just a few of them being informative and interpretable. My plan is to validate the estimator using 5-fold ...
20
votes
11answers
4k views

Does machine learning really need data-efficient algorithms?

Deep learning methods are often said to be very data-inefficient, requiring 100-1000 examples per class, where a human needs 1-2 to reach comparable classification accuracy. However, modern datasets ...
0
votes
0answers
7 views

Multiple models walk forward validation ARIMA

I am trying to build a robust model to predict attrition rates based on 88 data points.I came across walk forward validation. I have the following understanding of it: We can either use the same set ...
-1
votes
0answers
32 views

Confusion with train, validate, and test datasets in machine learning

I understand the theory behind splitting your dataset into 3 parts (train, validate, and test). However, I'm having trouble understanding if I'm implementing it correctly. From my understanding using <...
1
vote
0answers
24 views
+50

Find a representative samples from an estimated distribution by KDE

I served a Neural Network model trained on a huge (timeseries) dataset. In production, I would like to monitor the newly received data and check if there is a drift in the features using K-S testing. ...

1
2 3 4 5
342