The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
1answer
10 views

Algorithms implemented in Java to measure multi-core CPU performance [on hold]

I've been using Linpack and SciMark2.0 algorithms to evaluate CPU performance. I found those here: ...
0
votes
0answers
16 views

What is Ideal Point Error (IPE)?

I am working on a rainfall forecasting study where I aim to compare my results against observed values or any other model. I have been asked to develop a new performance measurement which is Ideal ...
0
votes
0answers
10 views
0
votes
0answers
12 views

Performance Tuning in R [closed]

I have a large Dataset with 150000 * 140 I have been trying to do stepwise Logistic regression with step(glm(x~y, data=data,family=binomial), direction="both"). It's taking more than 2.45 Hours ...
0
votes
0answers
8 views

What is a better approach to create operational performance score, LSM or PCA?

I want to create one operational performance score for each day so that we know if the performance was good or bad in our call centers, to be able to compare days, and to be proactive in maximizing ...
0
votes
0answers
11 views

Selecting a network based on Validation performance or test performance? [duplicate]

Sorry for repeating this question but honestly I could not find the exact answer for that.There are lots of opposite answers and solutions. If the dataset has been divided to 3 subsets ...
0
votes
0answers
17 views

External validation of a group's machine learning process

Say there's a group (that is opaque to you) that's heavily using machine learning methods to produce some outputs. You don't have access to their input data or code but can ask high level questions ...
0
votes
0answers
25 views

Variable that discriminates but performance metrics don't improve

I have a score adjusted by a logistic regression done with variables let's say X1 to X5. I'm not including a variable X6 to be chosen. Now, if I let the algorithm (stepAIC from R) choose among all of ...
2
votes
0answers
24 views

What would be a proper statistical test for multiple models and model selection?

Suppose I have a data set of N observations (n = 1...N) for out-of-sample estimation and values of ($y_n$). I have also ...
0
votes
0answers
19 views

My predictors have strong collinearities, yet linear regression performs as good as partial least squares. Why?

I am trying to predict a single response from twelve explanatory variables. There exist strong correlations between my variables. The correlation matrix looks as follows, and the data have a ...
0
votes
0answers
4 views

Ranking districts (on basis of % cahnge) after controlling initial conditions

We want to rank Indian districts as good/bad/average performer on the basis of change in access to latrine between 2001 and 2011. We calculate the percentage point change in access to latrine ...
2
votes
2answers
82 views

comparison of SGD and ALS in collaborative filtering

Matrix factorization is widely applied in collaborative filtering, and briefly speaking, it tries to learn the following parameters: $$\min_{q_u,p_i}\sum_{\{u,i\}}(r_{ui} - q_u^Tp_i)^2$$ And we could ...
0
votes
0answers
69 views

Can I add the scores of three principal components

I wish to create one operational performance score for each day so that we know if the performance was good or bad, to be able to compare days, and to be proactive in maximizing performance. I did ...
0
votes
0answers
13 views

Performance metric for one-class SVM classifier with small number of instances

I am using LibSVM one-class classifier for detecting a particular hand gesture from accelerometer data. I have trained two categories of classifiers: (a) user-dependent: data from the same user is ...
5
votes
1answer
60 views

Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
2
votes
2answers
78 views

R performance limitations [closed]

I attempted to predict crime category from its X and Y coordinates based on San Francisco Crime Statistics (https://www.kaggle.com/c/sf-crime). And it turned out that glmnet is pretty slow for fitting ...
0
votes
0answers
12 views

Comparing single-value and probabilistic online predictions

Let $\mathbf{x}^{(T)} = \{ x_1, x_2, \cdots, x_T \}$ ($x_t \in \mathbb{R}^d$) and $\mathbf{y}^{(T)} = \{ y_1, y_2, \cdots, y_T \}$ ($y_t \in \mathbb{R}$) be two time series. If I had a ...
0
votes
0answers
10 views

Compare accuracy of a new light meter

I'm performing a study where i want to test the accuracy of a new light meter (A) againsta a standard calibrated and validated light meter (B) as a reference. I have measure different light ...
0
votes
0answers
67 views

Year on year model performance evaluation via bootstrapping

I am interested in monitoring the yearly evolution of model’s performance (a Loss Given Default model more specifically). The yearly additional data, however, is very limited (maximally 5 ...
0
votes
1answer
33 views

Compute accuracy of model evaluation

Say the number of negativ classes is $9990$ and the number of positive classes is $10$. If a model predicts all examples to belong to the negative class, how accurate is this prediction? So what we ...
0
votes
0answers
20 views

In the classification framework, is AUROC a performance measure or metric?

I guess, the title is self-explaining. I have seen both so far and was wondering if there is a correct term or whether it does not really matter.
1
vote
0answers
14 views

Well defineness of the definition of F1 score

The F1 score is defined as the harmonic mean of precision and recall: $$ F_{1}=2\cdot {\frac {\mathrm {precision} \cdot \mathrm {recall} }{\mathrm {precision} +\mathrm {recall} }}.$$ The above ...
0
votes
0answers
19 views

Matched pairs versus multiple t-test measurements for task performance over time

I am trying to compare performance on a task over time across one group based on initial performance, but am running into a little trouble. As background, I am examining a group of students' ...
3
votes
2answers
37 views

Creating a rank on performance and size

Ranking is not something that I need to do particularly frequently in my job and its probably the weakest part of my statistics background. I have a dataset containing a list of items and a ...
0
votes
1answer
87 views

How to compare classification methods in terms of performance?

I'd like to compare logistic regression to classification trees. In a first step, I compared the theoretical framework of the two classifiers. In a second step, I compared the performance using a ...
1
vote
0answers
87 views

Why MSE of this MATLAB neural network is so high?

I'm using this code to train a neural netowrk in MATLAB: ...
1
vote
1answer
66 views

Is it valid to get better performance in logistic regression using only a subset of the coefficients?

I have an imbalanced data set containing 12% of the positive class 88% negative. First, I ran a logistic regression with all my coefficients and got an average accuracy of 0.91 (I know that's not ...
0
votes
0answers
17 views

MLP and RBF performance assessment (Matlab)

I hope you really help me. I'm now approaching to the world of data mining and pattern recognition and I need a little help. I have a data set composed by 4 struct of features. Each struct contains 18 ...
1
vote
1answer
22 views

How to account for games and number of teams in share of points?

I am doing research on the relationship of income inequality on team performance in soccer. The team performance I measure through share of points, so the number of points a team got in a particular ...
0
votes
1answer
15 views

Performance of estimators

I have 11 different estimators and try to figure out which of them performs best. The regression line between given and estimated values ideally should have a slope of 1 and an intersect at 0,0; which ...
0
votes
0answers
37 views

Model stability- ratios in same bucket should keep constant over different population?

I need to explain the variation in a validation table for a propensity score to buy insurance. I trained in R a logistic model with 4 months of 2014. Calculated limits of non-overlapping deciles of ...
6
votes
2answers
121 views

How can I assess performance of a semi-supervised learning method?

I'm working with a semi-supervised learning task, where I only have positive and unlabelled data (PU learning). I've tested a few algorithms and would like to assess their performance. For ...
0
votes
1answer
34 views

Best performance evaluation measure for GARCH model? [duplicate]

I want to compare the performance of various volatility models like GARCH, eGARCH, and gjrGARCH from actual volatility (computed using high frequency data). I found three common performance evaluation ...
0
votes
1answer
56 views

How can I compare two mutual funds' performance with a sparse set of data?

I want to compare the performance of two mutual funds. The only data I have is annual returns for the past 7 years. So I have 7 observations for Fund 1 and 7 observations for Fund 2. In addition, I ...
0
votes
1answer
73 views

Calculate thresholds of factor analysis output to classify data to 5 classes

Suppose that we calculate a composite indicator for some companies using Factor Analysis (FA) by combining five features to one output (calculate weights of input features). This is histogram of ...
0
votes
0answers
11 views

How does one show that an extension of an algorithm performs as well or better than the original?

If we have an extension of an algorithm (for example, we take OLS and scale the inputs, and solve it as a reformulated OLS problem), how does one show that the extension performs as well or better ...
0
votes
0answers
12 views

bootstrap simulation for CI

I have generate 4 functions to test the performance of my bootstrap for obtaining 95% confidence intervals on the mean of a univariate sample of data. However, it does not run properly with the last ...
0
votes
0answers
22 views

Performance Evaluation of Search Engines Test

I have to apply double-blind testing method to evaluate the performance of any two search engines.(ex. Google vs. Bing) I'm drawing a blank here. Anyone have any ideas of how I can start?
0
votes
1answer
52 views

Reporting result of classifiers [duplicate]

I am using a binary classifier in my research. I wanna report the obtained result of classifier, but I am not sure about it. The learning prosedure is as follows: ...
0
votes
0answers
20 views

Does this type of diagram showing conditional mean have a name?

I'm trying to show the skill of a prediction $\hat{Y}$ relative to a true value $Y$ beyond simple statistics like MSE or $r^2$. I usually like looking at the scatter plot of $\hat{Y}$ vs $Y$ but in ...
0
votes
1answer
63 views

GBM Performance on different sampling techniques

I am working on a healthcare data set for breast cancer patients. This data set is class imbalances and the distribution of positive and negative classes is 80%/20%. In order to deal with the class ...
0
votes
0answers
30 views

Uncertainty assignment to genetic algorithm solution

I have a black-box function for which I need to find its global maximum. I do this via a genetic algorithm (GA) since I can't solve it analytically. Since the GA can't guarantee to return the true ...
0
votes
1answer
73 views

Which is better, MSE or classification rate, for minimizing the likelihood of overfitting?

During the process of selecting a classification model (with cross validation), if one uses MSE of the predicted posterior probabilities as a selection criterion, then a model ...
1
vote
0answers
25 views

Standard performance measure for regression?

I will perform time-series prediction and I will report the accuracy of my system with a measure like RMSE or MAE. However, the variables I will predict are in different ranges. So let's say one is ...
0
votes
0answers
13 views

Term for results being similar in relation to each other, but differing is absolute magnitude

I have some results of several methods, applied to two tasks. Roughly they are on the lines of Task1 Method A: 20% Method B: 25% Method C: 40% Method D: 70% Method E: 71% Task2 Method A: 40% ...
0
votes
0answers
10 views

Performance metrics, prioritizing based on impact

I apologize in advance for the poorly worded question. If anyone has any suggestions, please advise. Also, help with tagging appropriately would be appreciated. I'm collecting performance metrics ...
1
vote
2answers
65 views

Visually summarising multiple diverse metric results

I have a multivariate time series (climate fluxes), and I have a number of model outputs that I want to asses. I also have number of metrics that I can use to assess the performance of each variable ...
0
votes
0answers
25 views

Performance measure

I am currently trying to show the trend of different team performances based on what percent of customer complaints they receive that they are able to handle within a given month. So if one team ...
2
votes
1answer
113 views

CART with Ordinal Response Variable using rpartScore Stuck

I'm trying to fit a decision tree over some data which has ~40K rows and ~200 features. The response variable, y, is ordinal and takes values ...
2
votes
1answer
47 views

What should the AUROC be on the test set when no positive example is present?

Assume we have a probabilistic, binary classifier. We compute the AUROC on a test set in which no positive example is present (i.e. the ground truth is always 0). What should the AUROC be?