**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

-1
votes
0answers
22 views

Weak classifier with Naive Bayes

I am trying to train weak classifiers from a specific data set, so I can boost them later on(with Adaboost). I have tried using Naïve Bayes as a weak learner. Here D1 is the expected output. Since ...
1
vote
1answer
21 views

Generating 2D different shapes with same mean and covariance matrix

Im trying to reproduce the synthetic data used in this article. The authors claim the data was generated from densities with the same mean and covariance matrix. Is there any principled way I to ...
-1
votes
0answers
30 views

SKLearn Clustering: how would you cluster a LARGE database of dogs? [on hold]

Given: A VERY large dataset of dogs. columns: ID (alphanumeric) Weight (numeric) Height (numeric) Eye Color (alphabet) ... (numeric) Tongue Length (numeric) How do you find what makes these ...
1
vote
0answers
12 views

Which is the dependent variable in the UCI Liver dataset?

https://archive.ics.uci.edu/ml/datasets/Liver+Disorders lists some papers which have used this dataset. I went to one, Turney, and it says: The target concept was defined using the sixth column: ...
1
vote
0answers
15 views

Simple Path to Route Algorithm [closed]

I'm currently conducting research into subway paths, but stand in front of a problem, with no programming knowledge and an extremely short time-limit. Probably the question I am asking is trivial, ...
0
votes
0answers
21 views

Are there any other websites which will let you compare worldwide stats for wide ranges of topics? [closed]

This is what I found after some googling and testing: http://www.tylervigen.com/spurious-correlations They only have U.S. data while I'd like to find worldwide stats. Anyway, should I need to ...
2
votes
2answers
21 views

Data set for document topic discovery

I have a bunch of documents and would like to to assign one (or more) topics to each document. The topics can be quite wide ranging from political topics to sports, etc. I think the best way is to ...
4
votes
1answer
31 views

Comparison of groups with low N

Recently, I have seen a study that was comparing three groups of subjects. Each of these groups had N=3 participants. The author was claiming very strong conclusions from the mean-differences of ...
0
votes
0answers
52 views

Data Preparation for Cluster Analysis

Updated answer to "Data Preparation for Cluster Analysis": Based on the discussions, data normalization and removing correlation among data are often recommended. References posts: 1) Are mean ...
1
vote
0answers
28 views

Finding patterns within inconsistent data

If I have a database full of data sets that I collected from a material sample going through light sensors. Each sample is similar but the data can be inconsistent depending on many factors. I ...
0
votes
0answers
9 views

Combining groups for data analysis

I have a sample of 8 participants randomly allocated to two groups. Each group has a different treatment. Initial analysis using non parametric tests shows no significant difference bewtween the ...
1
vote
0answers
24 views

Which test(s) to use?

I’m having a dataset that needs to be analyzed but I’m a bit worried about picking the appropriate test(s). It’s been a while since I last really used statistics and SPSS. Therefore I would kindly ...
1
vote
0answers
17 views

what possile balancing method can I apply on a imbalanced data set? [duplicate]

Im trying to solve one classification problem from de UCI database repository. https://archive.ics.uci.edu/ml/datasets/Student+Performance Unfortunately (or fortunately) I've noticed that my dataset ...
0
votes
2answers
32 views

US Population Hypothesis Test

I’m hoping someone can give me some tips on the test I have in mind for my data. I have data which is samples of Americans from various zip codes throughout the US. In my data I assume if someone has ...
0
votes
1answer
21 views

What is the problem with statistical outlier detection approaches if we have distribution of attributes?

A group of outlier discovery methods are statistical approaches. Two drawbacks mentioned for statistical methods in many books and papers: They can apply just on a single attribute We need to know ...
0
votes
0answers
21 views

Determine if huge data sets come from the same distribution (R or Excel help)

I am doing a work assignment where I need to determine if three data sets come from the same distribution. After plotting the histogram they seem to be some rendition of a lognormal. I currently ...
0
votes
1answer
16 views

Normalising extreme items within datasets

I have a a dataset where each item is a % above or below 100% (taking an individual item, dividing by the mean). In order to produce a rank I weighted each item by a % (summing to 100%) to provide a ...
2
votes
0answers
7 views

Changing Value of Macro Variable inside SAS macro [migrated]

I am defining a macro variable inside a macro. Then, I am feeding it into a second macro. Inside macro2 counter changes value to 200. However, when I check what is inside the macro variable that I put ...
0
votes
0answers
19 views

Equivalent of (sic) for data

When quoting some text containing some surprising assertion or spelling, one can use the latin adverb sic (sic erat scriptum) to indicate that the written text is exact and not an error of ...
0
votes
1answer
34 views

Statistical Significance

I have two datasets consisted of 4 columns. I have to measure if the differences among the data (each column represent another data) are statistically significant. The problem is that: in the first ...
0
votes
1answer
270 views

SVD interpretation of Climate data

I was going through my homework. I have a question to interpret the plot which i obtain from first and second left singular vectors after computing SVD. What can i exactly interpret, is it specific to ...
0
votes
0answers
16 views

Bagging Improvement for classification problems

Breiman proved that bagging improves the accuracy of regression. How could this be proven for a classifier model?
0
votes
0answers
21 views

How to learn from dataset where there are repeated instances but with different values?

I'm facing an issue with my dataset. The issue is that I have for example 1000 instances (customers or students) but I have 4000 lines/records because some of the instances are presented in several ...
0
votes
0answers
9 views

Which supervised algorithms benefit from knowing future inputs upfront?

I am making an open source learning library, QuantumLearn. Since I have all the freedom how to design the api, I was thinking that the fitting method could accept an arbitrary number of unlabeled ...
1
vote
0answers
10 views

How to data mine google for most popular things out of a set?

Let's say I want to get the list of the top 5000 most searched biomolecules on google - how would I approach such a problem? One possible solution is to fetch the whole pubchem/pdb database and run ...
-1
votes
1answer
16 views

How to statistically categorize a list of reasons?

I am working with call center data, one of the variables available is "Reason" which is a description of the reason the customer called. There is 40 different reasons that the agent can choose from. ...
0
votes
0answers
27 views

Visualize multivariate data in Excel

Here is an example of the data I want to visualize either as stacked bar or as scatter plot. ...
1
vote
2answers
17 views

Applying Akaike Information Criterion on collection of Gaussian fits

I am trying to apply Akaike Information Criterion on a collection of Gaussian mixtures fitted on some data points. My question is, can I use AIC even if the number of components of Gaussian mixtures ...
0
votes
0answers
18 views

How to build kpi on employee<>customer<>sales

For further analysis i want to build a KPI which describes the "critically" of a employee<>customer relationship. To make it clear, some example data: Employee - Customer - ...
0
votes
1answer
50 views

Daily data, different dates

I have downloaded two different daily data sets, both for the same period. Together I want to use them to construct data for another variable. However, the problem is that the dates do not exactly ...
0
votes
0answers
23 views

Data set sample

I'm willing to make a comparative study, to examine culture dimensions effect on countries with the highest GDP and countries with the lowest GDP countries, but there is no enough observation to make ...
4
votes
1answer
162 views

Data augmentation techniques for general datasets?

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, ...
2
votes
0answers
35 views

Is it possible to explain the performance of a classifier by the specific properties of a data set?

I receive the following comment from a reviewer : "I think that the authors could explain in more details the results. For instance, there is no discussion linking the specific properties of the ...
0
votes
1answer
43 views

Is building training data set from unlabeled data considered as a scientific contribution? [closed]

Is building a training data set, from unlabeled data, for a machine learning classifier considered as a scientific contribution?
5
votes
2answers
124 views

What is bucketization?

I've been going around to find a clear explanation of "bucketization" in machine learning with no luck. What I understand so far is that bucketization is similar to quantization in digital signal ...
0
votes
0answers
10 views

How compare aggregate data before and after policy?

I have aggregate data on the number of tests performed at the laboratories of two hospitals and two departments each, before and after insertion of an information system (data refer to costs and ...
2
votes
1answer
27 views

How to process Binomial-Binomial data on SPSS for hypothesis testing?

There are 2 groups of 100 pregnant women: the first are malnourished and the second are well-nourished. A COHORT research is being done on them to determine if malnutrition is a risk factor for low ...
1
vote
2answers
66 views

How to Balance my Dataset?

I have 90% negative examples and 10% positive examples,(13,000 observations, 90 Variables). my model shows me that the miss classifications error is 0.1 but my confusion matrix shows me that the TP ...
0
votes
0answers
41 views

What test to use? Splitting up dataset -> increase performance value

I am working on my thesis and I have collected some data. Now I need to know what kind of test I should use. I have an algorithm with which I produce a bunch of data over a dataset. Then I calculate ...
0
votes
0answers
13 views

Minimizing information loss when aggregating transaction data

I have rich data set of transaction data as follow: ...
2
votes
1answer
91 views

What should be my training set for a word-spotting program?

I wanted to create a simple voice recognition program. It should be able to distinguish my voice from other sounds. Aside from my voice, what should I add to my training set? Also, I want to extend my ...
1
vote
0answers
22 views

Standard deviation comparison for splitting clusters in ISODATA

I am currently implementing the ISODATA algorithm and I am new to cluster analysis as I just learnt about it. I got stuck at the step which I need to compute the standard deviation of each cluster, ...
0
votes
0answers
14 views

Should I deflate capital stock with price index?

I am working on productivity growth and y(value added) is my dependent variable and k(capital stock or fixed asset in CMI) and L(labor) are independent variables.I have deflated y and L with whole ...
0
votes
0answers
19 views

How to determine the number of employees in a business when sources are inconsistent?

I am analyzing a business survey data. I am interested in the total number of employees of each firm. Specifically, I am interested in: The total number of employees in a specific city, and The ...
1
vote
0answers
28 views

Should I use A/B testing to find evidence that click-through rate has changed over four months?

I am working with a toy data set. I have some data for a website over the last four months. This data includes how many times (and when) an ad was shown and how many times an ad (and when) was ...
1
vote
0answers
32 views

Research question about single-group interventions and data analysis

We are working with a community organization that had collected data on their own and want us to do data analysis. The data they sent us showed they gave everyone their (anti-smoking) intervention and ...
0
votes
0answers
11 views

Usual way of measuring a set's deviation from another set

I have a set of reference daily prices. I also use an algorithm to estimate daily prices. I want to measure the difference between my estimations and the reference. I am quite unfamiliar with ...
0
votes
0answers
46 views

Which algorithm fits best to a transactional data classification problem?

I have transactional database - the data looks as follows: ID - COLUMN1 - COLUMN2 - COLUMN3 0 - A - B - C 0 - A - D - C 0 - E - B - C 1 - A - B - C 1 - A - B - C 2 - ...
0
votes
0answers
19 views

Structuring my Survival Model Dataset - Dealing with variables that change

I am working on a survival model where I am interested in how many hours a patient will stay in a hospital. Therefore, the event is Discharge from Hospital. One of the variables I would like to ...
3
votes
1answer
87 views

What's the meaning of dimensionality and what is it for this data?

I'm doing my assignment for my "Modeling and Optimization" course, and now I have doubts on the first question: What is the dimensionality of the data? What are the min, median, max, mean, ...