**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

0
votes
0answers
11 views

How can I add an ID variable that gives each set of duplicate rows a separate label? [on hold]

I have data that contains many rows with eight identical columns. I would like to group the rows that are identical and assign a unique ID variable to these groups.
1
vote
0answers
11 views

Guessing/extracting from the pool of partially right guesses

Lets assume that I have integer numbers from 1 to 50. I imagine any 5 unique numbers from 1 to 50. A computer can generate an array of 5 random numbers, as many times as it wants. Each time it ...
1
vote
0answers
4 views

Open datasets for educational use [migrated]

I'm interested in compiling open datasets for educational use. For example, this page from the State University of New York Geneseo includes open datasets by topic. I'm especially interesting in ...
0
votes
0answers
19 views

Correlation for cases of disease per year?

In my data set the y variable is the year (as in 2015 or 2016) and the x variable is the number of HIV cases. I want to do a simple correlation on this data set to see if HIV is increasing or ...
0
votes
0answers
6 views

Filter only names of Cities from datasets [on hold]

I have a list of keywords and they contain several cities names and countries. The data set contains several other keywords within but I want to filter only the geographical names from the list. I am ...
0
votes
0answers
5 views

achievable accuracy in common datasets

I have been using the credit approval dataset https://archive.ics.uci.edu/ml/datasets/Credit+Approval and I have achieved an accuracy of about 86%. I want to know how good this is. Is there a ...
0
votes
0answers
11 views

Aggregating Dataset with many categorical variables [closed]

I have this dataset wich is structured like this ...
0
votes
0answers
14 views

Reporting Data with Estimates and SE

What do you think? When reporting data in an article, a usual way is to show the mean values +/- the standard errors in a table. For Example, when you measure Abundance of pigeons in parks of five ...
1
vote
1answer
48 views

Probability that the age is a multiple of 10

I am working on a data set, containing demographic informations in a developing country. I noticed that a third of the ages is a multiple of 10 ! I expected (though the distribution of ages is not ...
0
votes
0answers
16 views

Data mining conclusion [closed]

I am undertaking a data mining challenge which was posted by the stack overflow website. I have reached the end of the project but am not able to understand the conclusion part. In the sense I have ...
1
vote
0answers
24 views

Looking for “Data gathering is hard and expensive”

I've finished some data mining and machine learning courses and in most of them at the very beginning has been said: "Collecting and pre-processing data is the most expensive and hardest part of the ...
0
votes
0answers
10 views

Useful inputs from files for neural network?

I'm building a visual neural network sandbox. ATM i'm working on the Input nodes, where you have a browser to choose a file from your drives and drop it in the workspace. The workspace detects what ...
0
votes
0answers
7 views

Controlling Data [closed]

I'm studying on outliers. The following codes are suitable for multivariate normal distribution. But I want to perform them for multivariate laplace and multivariate cauchy. How can I write? Changing ...
0
votes
0answers
22 views

transformation in logistic regression model

So we know we can use BoxCox or inverse transformation in linear regression or continues dependent variable models. you can use boxcox() invResPlot() or powerTransform() in R for transformation your ...
0
votes
0answers
10 views

How to deal with skewed categorical field? [closed]

I have a data set and I have 28 predictors and 1 target variable. When calculating predictor importance, the most important field which I have been able to find is skewed. So, should I consider that ...
0
votes
0answers
9 views

Topics needed to learn to for predicting future of a data set?

I am planning to work this summer by writing a code which will eventually become an app, that has the ability to predict what the prices will be atleast 6 month ahead of time. I have a data set of the ...
1
vote
0answers
32 views

Is the dataset Normally distributed?

So based from this link http://www.simafore.com/blog/bid/107702/2-ways-of-using-Naive-Bayes-classification-for-numeric-attributes I began to realize it might be a good idea to compute the pdf of ...
0
votes
0answers
11 views

regarding generating synthetic data simulating the real data

We are trying to develop some predictive models. The current scenario is that we have to rely on synthetic data at first since the real data set will not be available quite soon. It is understandable ...
1
vote
0answers
18 views

Which model(s) are appropriate for this kind of data

So, I tried to implement a model on some data. The dependent variable is a ratio that can get higher than 1, is lower bounded by zero and, seeing figure 1, is left skewed.Thus, a logit regression is ...
0
votes
0answers
15 views

Numerical Similarity Measures for Dirty Data

I need to compare two datasets to see if they are "similar" in terms of numerical similarity between given columns. The problem is, however, that I do not know the characteristics of the 2nd dataset. ...
0
votes
1answer
116 views

How to graph throughput

UPDATE 2: maybe a simple way to state it: for web application performance data consisting on: start timestamp | end timestamp | response time I'd like to compute a non-aggregated data set consisting ...
0
votes
0answers
8 views

Creating subsets of data with a particular target distribution

Is it possible to select a subset of observations from a dataset, while enforcing particular distributions across the data dimensions? In essence, create subsets of data with particular target ...
0
votes
1answer
11 views

Absolute Average deviation in percentage calculation

Sorry if my terminology is incorrect. I am trying to calculate the average error of prediction to be represented in percentage. For example, I should be able to say, the predicted values are on ...
2
votes
1answer
62 views

Random effects model for a derived variable - how to use the data

My data is a series of measures for a biological trait ($Y$ - e.g. weight, height, length, running speed) from multiple populations ($n$ = 2), taken over a number of years, in males and females of a ...
0
votes
0answers
25 views

No change in accuracy big vs small training set size ConvNet

I am doing some small experiments with image classification in Caffe using the AlexNet architecture. I use a dataset of 50 classes with each class containing 1,000 training images. After about 2k ...
0
votes
0answers
13 views

Is resampling more accurate than block average for statistical analysis of data (cross-post)?

this is a cross question I asked on the computer science community before I was advised to cross-post it here. I'm working in laboratories where molecular dynamics data are almost always analysed ...
1
vote
0answers
7 views

Creating products overall score across different data sets

I've created product score mapping that gets ranking for product according to reviews from different websites and in different contextual categories. There are 3 contextual categories: eCommerce/ ...
0
votes
0answers
12 views

Bird eye view of unknown data set [duplicate]

Pardon me if this question does not fit this site (or SE engine in general) but I do not know where to post it. Suppose that you have unknown data set which you want to explore. What ...
0
votes
0answers
7 views

Analyzing Average Trend with not matching years

I am working on analyzing census data. Most census data are 5-year average (year 1995 means average of 1991-1995, year 2000 means average of 1996-2000, etc). I am trying to match with other datasets ...
0
votes
0answers
11 views

nonlinear time series analysis

i need to forecast daily stock prices using a number of econometric variables. stock price data is high frequency and the independent variables such as exchange rate, industrial production, money ...
0
votes
0answers
23 views

Get most occurring value on a graph

I have a data series which is continuous (there is data in between individual data points not captured by the series). There are many inflection points within the series and my goal is to find 5 most ...
0
votes
0answers
14 views

Find sample size representative of data

I have data from multiple farms, showing how many healthy and lame animals are found on each farm, and how many animals were there in total. I would like to calculate, statistically, how many animals ...
1
vote
0answers
19 views

How to create a realistic multi-label data set of web pages? [closed]

I am working on a project which is a web filter. The web filter is supposed to classify web pages using any multi-label classification technique and then block them if anyone of the category is found ...
0
votes
1answer
15 views

What tests to analyze the following hypotheses with nominal data?

I have a set of questionnaire data to analyze the working culture for respondents. (There are four existing cultures; a,b,c and d) Variables including Gender, Organization, Public/Private Sector, ...
1
vote
0answers
8 views

Find (latent) variables which correlate with age of samples (Time series correlations)

I have 8 samples of whisky which are all from the same producer, matured in the same type of casks. However, the casks were (naturally) different. My ages range from 5 years to 32 years old, but ...
1
vote
0answers
31 views

Student’s T-test

I am currently doing research using Matlab where I am looking at a variable in convective weather (i.e. 200 values of CO2 content) and comparing it to the same variable with a different length in ...
1
vote
1answer
32 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
0
votes
0answers
4 views

Which efficient algorithms for calculating marginal counts over a complete dataset

Having a complete dataset over a set of discrete variables, I need to compute the marginal distribution over different subsets of variables (For instance all subsets comprising 3 or less variables). ...
1
vote
1answer
38 views

What does positive skew show?

After collecting a sample my data is positively skewed, what does this show? I have collected data on the price of books and I have found my data to be positive skewed however I am not sure what this ...
1
vote
1answer
63 views

How to develop a dataset for the gravity model of international Trade

The gravity model of International trade is used to estimate the determinants of bilateral trade between countries. In developing the dataset for the gravity model, do i need to manually pair the ...
0
votes
0answers
9 views

dataset construction

I am trying to build a dataset for benchmarking but for which the sizes of each group is quite different. I am feeling there is something a bit dodgy in my current methodology but cannot point it out. ...
0
votes
0answers
42 views

Multiple imputation for predictive analysis using mice package in R

I am using the mice package to impute some missing values, and it works nicely. I am facing a tricky strategic question though. Basically, I am working on predictors of myocardial infarction (time ...
0
votes
0answers
16 views

Multiclass classification with large number of classes but for each user the set of target classes is known

I'm new here but I'm familiar with some machine learning theory (took some courses in school) and my question is more about how to apply ML in a practical setting. I have this project where I'm ...
1
vote
0answers
27 views

How to interpret a dataset with a pseudo-logarithmic trend?

I am dealing with a scatterplot where I am trying to figure out the relationship between two variables, but I have so far failed to identify what the best fit could be. The presence of ...
1
vote
0answers
10 views

Merge sample data into dataframe [closed]

I have n samples for which I have acquired mass spec data. The output of this is now n CSV files, each containing a list of (strings) formula and associated abundances (floats) for each peak in the ...
0
votes
0answers
25 views

Logistic regression matrix within array

Im trying to learn logistic regression and im confused while trying something So for example this is not the real data but it looks something like this so I have multiple images and save their rgb.. ...
1
vote
0answers
39 views

Calculating Conditional Probability in Real Data

I have my data in the form as: ...
0
votes
0answers
12 views

How to model/predict discrete curve locations from numerical and categorical independent variables

I want to find the principal "parameters" driving the relative location of a set of $K$ discrete curves (observables, dependent variables). The data shape is as follows: ...
0
votes
0answers
11 views

Probability of equal indicators

Currently I am working with commodity price data that also offer closing price, opening price, lowest and highest price throughout the day. Curiously, I encountered one day that had the same values ...
1
vote
0answers
39 views

handling missing input variable for machine learning

In building feature-sets for a machine learning algorithm, I'm facing a situation where the input variable - which is a numeric variable, may or may not appear. What I mean is, the data set I'm ...