Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
1
vote
1answer
13 views

Can Neurons (Features) be repeated in Dense Layers

I was revising Convolutional Neural Networks and encountered the following question. If I were to classify a cat and a dog (the famous cats vs dog classifier), then assuming there are 2 Dense Layers ...
0
votes
0answers
8 views

How to find the outliers(trending data) for real estate data

I have prices, location, type, rooms, etc info about various houses from a real estate website. I want to find the trending data or the houses which are unique, e.g. a house in New York, with 2 rooms ...
0
votes
0answers
13 views

How would you find a p threshold for a binary classification prediction?

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
0
votes
0answers
17 views

Overfitting: how to spot it in cross validation, confusion matrix and contingency table?

I am trying to understand if my results are overfitting or not. I have the following results, using different features for model building: Model 1 ...
0
votes
1answer
22 views

Discrete KL Divergence with decreasing bin width

I'm familiar with the definition of the KL divergence between two discrete distributions $D_{KL} = D_{KL}\big({\it P}(A) || {\it Q}(B)\big)=\sum_{j=1}^{n} {\it P}(A=a_{j}) \log \Big( \cfrac{{\it P}(A=...
0
votes
0answers
22 views

Theory Question: Machine Learning & Feature Correlation to Label

I have a theoretical question about creating an artificial feature based off of a binary classification label, and then adding it into my feature set to run my analysis. First, let me show you what I ...
2
votes
0answers
23 views

How does a Gaussian Process define a probability distribution in the functions space?

I am studying Gaussian Process Regression. I will post a text from the book Gaussian Process for Machine Learning, by C. E. Rasmussen & C. K. I. Williams: We first consider a simple 1-d regression ...
1
vote
1answer
14 views

Is SVM uses latent variables when input variables/features are superior than 3

I was wondering if SVM uses some kind of latent variables / latent space when inputs variables/features are superior than 3. In fact I know that SVM uses dimension - 1 (a curve in 2D, a plane in 3D, .....
1
vote
0answers
14 views

How would you approach this censored data problem? Time series or binary classification or other?

The context I have a financial dataset containing variables used to predict if a person will accept loan offer or not, we use a binary variable to indicate that (0 for not accepted, 1 for accepted). ...
1
vote
0answers
18 views

Proof of unbiased estimator

Assume: $$ \phi = \int f(x)p(x)dx = E_p(f)$$ Let $x_s \sim p, s=1,.....,S$ iid $(p(x_s=x)=p(x)$ and $p(x_1,x_2) = p(x_1) p(x_2)$. \begin{align} \hat{\phi} &= \frac{1}{S}\sum_{s=1}^{S}f(x_s) \\ E[\...
1
vote
0answers
7 views

I am curious on “ how to 'grey out' certain output units in neural network depending on different observations?”

Typical code examples I have found does something like this : Feature Engineering e.g. One Hot Encoding, Label Encoding etc in pandas data frame. Splitting into X and y [as numpy array] Defining ...
0
votes
0answers
17 views

R: measuring unusual answer patterns in a questionnaire

I am often dealing with online questionnaires and I have to identify respondents that do not fill out the questionnaire seriously in order to improve data quality. In order to identify "speeders&...
1
vote
0answers
18 views

How to add labels to an already trained Yolo model?

I'm learning ML and I'm exploring object detection and classification. I discovered Yolo few months ago and it's impressively efficient and accurate. There are several pre-trained Yolo models on the ...
0
votes
0answers
22 views

Is it a bad idea to always standardize all features by default? [duplicate]

Is there a reason not to standardize all features by default? I realize it may not be necessary for e.g., decision trees but for certain algorithms such as KNN, SVM and K-Means. Would there be any ...
0
votes
1answer
36 views

How can I code the sequence of [1,2,3,4,5,6,6,5] in 2.5 or 3 bits?

If I calculate the entropy for the following sequence: [1,2,3,4,5,6,6,5] I get the entropy of 2.5 but I am wondering how can I actually do the encoding with 2.5 or 3 bits. Does it mean I need 3 bits ...
1
vote
1answer
12 views

Are all predictors assumed all dependent to each other for a decision tree model?

I am wondering if all predictors are assumed to be dependent to each other when building a decision tree model. For example, a naive bayes' classifier assumes all predictors are independent but I am ...
1
vote
0answers
23 views

Inconsistent ROC-AUC

I'm training VGAE to predict edges and I'm getting a good loss (BCEwithLogits) values but the ROC-AUC seems bit inconsistent and not very progressive. I'm not sure how to diagnose it. The model is ...
0
votes
0answers
11 views

Interpretation of Wald-Wolfowitz Runs Test for detecting Covariate Shift

I have been looking at several documents about applying statistical methods in R for detecting covariate shift in machine learning models. A few articles have recommended using the runs test for this ...
0
votes
0answers
10 views

Search optimal parameters on predictions for a trained machine learning model

I have created a ML model from the following electric circuit As the imput I'm only using the values of R3 and R5 which I'm changing an the ranges: R3: 1 to 5 R5: 10 to 150 in steps of 10 My target ...
0
votes
0answers
6 views

How to interpolate and extrapolate using hte smooth.spline function in R [migrated]

Hello I currently have the following code to generate a smooth spline and was wondering if anyone know how to interpolate and extrapolate using this. For example I would plug in an x values and I ...
2
votes
0answers
18 views

Is it always better to use the RobustScaler (vs StandardScaler)?

From reading the docs, I believe the RobustScaler is more immune to outliers that the StandardScaler. In that case, why not just use the RobustScaler always?
4
votes
1answer
102 views

Is gamma actually an efficient way to weigh future rewards in reinforcement learning?

Typically the discounted sum of rewards is defined as follows: G_t = Sum(gamma ** n * reward_t...) But this means that rewards are worth exponentially less with ...
2
votes
0answers
16 views

Partioning train and test data in a time series

I have a monthly time series data from 2013 to 2020, Is it safe to do a fixed partioning where I train on 2013 to 2018 then use 2019 to 2020 as my test data? I am just scared that the will be a data ...
3
votes
1answer
21 views

Expected square error formula

I am going through this blog post to understand the Variance - Bias Tradeoff. In this, the author mentions: Err(x) = E[(Y - f'(x))^2] I understand this. But then, ...
3
votes
1answer
40 views

Bayesian Survival Analysis - COX PHD Time Varying Covariates Implementation

Suppose you are interested in the Survival modelling technique Cox Proportional Hazard, where we model the hazard as: $$ \lambda(t \vert x) = \lambda_0(t) exp (\beta x) $$ An extension of this model ...
0
votes
0answers
19 views

Which Random Forest hyperparameters to tune with Grid Search and which are the best initial hyperparameters values? [duplicate]

I want to use Grid Search for finding optimal hyperpameters for Random Forest. My questions are: Which Random Forest hyperparameters are considered important for tuning? Which initial Random Forest ...
1
vote
2answers
36 views

Grid Search using strategy

What is the correct strategy of using Grid Search? Am I understand correctly that to use correctly Grid Search I should: Give Grid Search initial parameters that have wide range. For example if ...
1
vote
0answers
10 views

Implementing Principal Component Analysis in Azure ML Studio [closed]

After applying Principal Component Analysis PCA to my data set in order to achieve better model accuracy. The 13 features dimensions, I am reducing it to 10 features using PCA. Everything is fine till ...
3
votes
1answer
41 views

Best approach for energy demand forecasting

I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this: ...
2
votes
1answer
37 views

How important is outcome variable scaling in SVM regression?

Should I scale outcome variable for SVM regression? What is the magnitude of impact of outcome variable scaling in SVM regression?
1
vote
0answers
20 views

Imbalanced class issue

I am taking my first steps in machine learning and data science area. I know for sure that my next task will be related to the imbalanced class problem. I’ve walked through many articles covering this ...
0
votes
0answers
16 views

Best technique for dimension reduction given binary and ordinal variables

I'm currently in the process of tagging a bunch of photos. I started off with some tags being binary variables (i.e. the tag in question was either present or absent) and some ordinal variables , ...
0
votes
1answer
31 views

How to generate a confidence interval for an adaboost prediction?

I have created a simple AdaBoostRegressor (sklearn) model which is trained on a feature set $X$ (house features) to predict a variable $y$ (house prices). The model can be used to create a prediction ...
0
votes
0answers
11 views

what's the output of RNN (LSTM) in the prediction of time series [duplicate]

It is a very basic question of RNN (LSTM). Here is the basic structure of RNN: for a sample $(X^i,Y^i),$ input: $X^i = (x_1,\cdots,x_T);$ output: $Y^i = (y_1,\cdots,y_T).$ $$z_t = Uh_{t-1} + W x_t + b,...
0
votes
0answers
13 views

Confused about the choice of the mean and the covariance of a function of random variables

I am very confused here. Let Z = (X,Y) be a random variable distributed according to a data distribution P_Z. Let K = XY be a random variable that depends on X and Y distributed according to a ...
0
votes
0answers
7 views

Mixed-effect logistic regression model and cross-validation

I using a mixed-effect logistic regression model and I need to do a cross-validation to check the accuracy of my model. I have 20 observations for each individual and 500 individuals in my dataframe. ...
0
votes
0answers
6 views

Can sombody interpret the explanation about Gap Statistic for Clustering?

The following content comes from the following site: [https://www.datanovia.com/en/lessons/determining-the-optimal-number-of-clusters-3-must-know-methods/][1] The algorithm works as follow: Cluster ...
0
votes
0answers
10 views

Latent space for cross domain features

I would like to find the shared latent space between two set of features. I have source and target domain features already extracted from images. I have 4 set of feature vectors for normal and ...
0
votes
0answers
11 views

The validation accuracy gets lower when the number of workers increases in Federated Learning with non-IID dataset

I use human activity recognition (HAR) dataset with 6 classes using federated learning (FL). In this case, I implement the non-IID dataset by assigning (1) each class dataset to different 6 workers, (...
2
votes
1answer
24 views

Deriving the gradients for Softmax logistic regression classifier

In the softmax logistic regression classifier, we have that $$\textbf{a} = W\textbf{x} + b\\[1ex] \textbf{z} = \text{softmax}(\textbf{a})\\[1ex] L(\textbf{z},\textbf{y}) = -\sum_k \log(z_k)y_k$$ In ...
2
votes
1answer
28 views

How do I validate a regression model's inferences and not its predictions?

Suppose over many years I collect data $X$ on a quantity of interest $y$ and some control variables $Z$. I fit an OLS model $$y = \beta X + \delta Z + \epsilon $$ and use the coefficients $\beta$ and ...
1
vote
1answer
17 views

Feature manipulation for classification model

I'm hoping to gain some advice in dealing with a classification model. Let's say I have three outcome labels [low, medium, high] and three features [F1, F2, F3]. To keep it simple, let's say the ...
1
vote
1answer
24 views

How LightGBM deal with a new categorical value in the test set

Suppose I have the training data set $(X, y)$ where $X$ is my feature space $(x_1, \dots, x_n)$. Let $x_1$ be a categorical feature column. In my test set, if the feature $x_1$ takes a new categorical ...
0
votes
0answers
13 views

SVD - vectors in matrix A

In SVD we have $A = U \Sigma V^T$. When applying it for ML, e.g. to calculate Moore-Penrose pseudoinverse for linear regression, I have seen that we take columns of $A$ as vectors. Typically in ML I ...
0
votes
0answers
15 views

VAR (Vector Auto Regression) with exogenous variables

Is it possible to set seasonality as dummy variables in a VAR model? Which I use those dummies as my exogenous variable?. Is it also advisable to input other variables that are a good measure of my ...
33
votes
6answers
4k views

Are neural networks better than SVMs?

For some time now I have been studying both support vector machines and neural networks and I understand the logic behind each of these techniques. Very briefly described: In a support vector machine,...
0
votes
0answers
10 views

Questions about mirror descent

I have 2 questions about the fundamental properties of mirror descent. Assume $D \subset R^n$ is an open and convex set, $X \subset R^n$ is a convex set, assume $D \cap X \ne \emptyset$, $X \subset \...
0
votes
0answers
25 views

Information Retrieval and Event Prediction from Unstructured Document Corpus

My question is quite open ended. In some chemical plant, by using the sensor data available we first deployed a machine learning tool that can predict the onset on anomalous behavior with some decent ...
0
votes
0answers
18 views

Effect of data scaling on model training

Standardization of a dataset is a common requirement for many machine learning estimators. If a feature has a variance that is orders of magnitude larger than others, it might dominate the objective ...
0
votes
1answer
20 views

I have doubts about descending gradient and backpropagation?

i am a beginner in this from ai and i am learning about gradient descent and its update rule Is it true that the same gradient is applied to each weight of the network at each step? that is, for ...