Structured data files in any format, collected together with the documentation that explains their production or use.

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

Data Validation [on hold]

I am doing data validation for customers with specific revenues; I have already received a Target group and a control group with revenue and number of customers; I want to validate the percentage of ...
4
votes
2answers
146 views

Best way to represent data

What is the best way to represent the following data graphically? Can I use a histogram ...
0
votes
1answer
25 views

Do you know of any public circular/angular dataset? [on hold]

I've been trying to locate and access circular/angular datasets for my research. I'm particularly interested in datasets that distribute according to a mean centered circular distribution with ...
0
votes
0answers
11 views

Term to Describe Strongly Clustered Data

I have some data which are strongly gathered into more than one cluster. I am looking for a term to effectively describe this phenomenon: e.g., multi-clustered data, which however seems to me that we ...
0
votes
1answer
45 views

How would a decision tree algorithm apply to the Mushroom dataset [on hold]

I'm looking for dataset to explore that would give me an opportunity to understand the pros and cons various ML algorithms; such as decision trees, nearest neighbor, SVM, and neural. I'm going ...
1
vote
1answer
37 views

statistical analysis with interval data

I have been assigned the task of helping someone who has collected some survey associated data related to some kind of government facilities provided to people. The data corresponding to small scale ...
0
votes
0answers
20 views

Categorical Variable with more than 50 levels

I am new to analysis. I would like to know relation between my 2 sets of categorical variables , Both my variables has more than 50 levels. Is it a good approach to create dummy variable and do the ...
-1
votes
2answers
34 views

How do you draw a random sample of unique IDs in a large dataset?

Rookie here -- I have a a large data set with about 75,000 observations, and 2000 unique IDs. Therefore, each unique ID has about 37 observations. I'm trying to draw a random sample of unique IDs, say ...
1
vote
1answer
19 views

Distribution comparison across different treatments

if I have a table like this In which, site = different sites of measurement (tissue or organ) type = different types of virus And I want to ask something about the data 1.What test can I use to ...
1
vote
2answers
16 views

clustering evaluation based on a gold standard

I have a dataset of items and I want to measure how my clustering method works. I'm using R and simple k-means clustering. I have a lets say gold standard of my clusters and I want to see how good is ...
0
votes
0answers
8 views

What is an equi-depth partition of the data?

In the paper Outlier Detection for High Dimensional Data at the beginning of section 1.3 Is written: Each attribute of the data is divided into $\phi$ equi-depth ranges. Thus, each range contains ...
0
votes
0answers
10 views

Telecommunication Billing DataSet

Has anyone seen any telco billing dataset available for analysis? I looked at Kaggle or http://www.d4d.orange.com/en/presentation/data but couldn't find anything.
0
votes
1answer
46 views

problems to build the linear multiple regression in R and to explain the output [closed]

I have a problem to build and to explain the linear multiple regression. I have a data set called Cars93 with 26 variables (numeric and not numeric) and 93 ...
1
vote
0answers
12 views

Longitudinal data format in WinBUGS or OpenBUGS, different number of items per time point

I am working with longitudinal data, where there are a different number of "items" per time point. I am working with the MCMCdynamicIRT1d function in ...
2
votes
2answers
29 views

Get the distribution of a dataset

this is a conceptual question. In data mining, the problem often arises that scientists/data engineers are using an expert guess for the underlying distribution of their data. Often their assumption ...
2
votes
0answers
18 views

Stores and ordered rank of products

I have data about popularity of some products in some stores. For each of the stores I have only a ordered rank of best-selling products in them. I need help how to find information about which ...
2
votes
2answers
29 views

Tool for generating correlated data sets

Does anyone know of a tool that I can use to generate a set of data with known correlations (and to put the icing on the cake - output this in json,csv,txt or some common format)? I am working on ...
0
votes
0answers
10 views

Using Cronbach's alpha to test reliability of a questionnaire

I administer a questionnaire to managers asking them to what extent they agree / disagree that they perform certain functions (listed in the questionnaire) on a scale of 1-5. The sample functions are ...
2
votes
0answers
22 views

Unbalanced Panel: pooled OLS vs FE vs RE - which method yield unbiased and robust estimators?

I am conducting an empirical study (relation between earnings and returns). I have an unbalanced panel with $N=449$ firms and $T=36$ time periods. Regarding the estimation methods I am confused: In ...
0
votes
0answers
32 views

compare strength algorithms across different data sets

Reflecting on this paper. The use of a statistical significance test is required in order to compare the algorithms across different data sets. In that paper and other papers, the number of data ...
0
votes
1answer
93 views

German credit data: neural network, svm, logistic regression : input variables

I'm using the following data set on some credit scoring models: https://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data) My teacher told me that it's best to use the same data set for all ...
0
votes
0answers
9 views

Find the 2 max values for each factor in R [migrated]

I have a question about finding the two largest values of column C, for each unique ID in column A, then calculating the mean of column B. A sample of my data is here: ...
0
votes
0answers
3 views

How to read this multi-format data in SAS? [migrated]

I have an odd dataset that I need to import into SAS, splitting the records into two tables depending on formatting, and dropping some records altogether. The data is structured as follows: ...
0
votes
0answers
15 views

Removing labelling noise

I have a big data set with unlabelled observations (several million) and about 20 thousand properly labelled ones. There are only two classes and all correctly labelled samples belong to the same ...
1
vote
1answer
19 views

Bidding Collusion Question: Probability of Identical Bids

I was asked this question. My initial conclusion is that there isn't enough information to calculate the probability. I would appreciate it if anyone could provide provide their insight. Scenario: ...
0
votes
1answer
31 views

determine independent and dependent variables without apriori knowledge of the nature of data

i have a spreadsheet with seven parameters and their values for 9000 samples.I need to determine which are dependent parameters and which are independent. Please let me know which algorithm might be ...
1
vote
0answers
13 views

Regression of irregular time series

I am beginning to look at high frequency data sets. I have only ever dealt with regularly spaced data up until now so wondering what best practice is. If I have two time series X and Y with Y being ...
0
votes
1answer
30 views

Simulate data by using existing dataset

I have a complete dataset with input variables and response variables. I would like to perform a simulation where I give the input variables and generate randomly the response variables. Is there a ...
16
votes
4answers
270 views

Good data example needed with covariate affected by treatments

I have looked at a lot of R datasets, postings in DASL, and elsewhere, and am not finding very many good examples of interesting datasets illustrating analysis of covariance for experimental data. ...
0
votes
0answers
45 views

Help on small data set with correlated features

My data set is quite small and feature rich. It has 30 observations and 30 features/predictors. In addition to that, many features are correlated (often not linear). I have tried several things- ...
2
votes
1answer
29 views

What do NORB and CIFAR stand for?

The MNIST dataset is a standard benchmark data set of digit images. MNIST stands for 'Mixed National Institute of Standards and Technology'. The NORB dataset is a commonly used dataset of binocular ...
0
votes
0answers
11 views

Invalid measurement data removing

I have a lot of data (temperatures and similar things) collected using automatic devices. Lots of these data is let's call it smooth: even when the temperature increases/decreases fast there are few ...
2
votes
0answers
37 views

Averaging z scores when doing a “group by”

I have a dataset where each row is an hourly measurement of certain fields (columns). For each column I then add another column that is its respective z score relative to the entire population. If I ...
1
vote
0answers
10 views

Derive minimum positive rate of change for co2 data

I have CO2 (in parts per million) data of a closed room. The CO2 data is recorded along with timestamps. Typical difference between two samples is around five minutes. My aim is to find occupancy of ...
2
votes
2answers
35 views

The size of the sample for split validation

At this moment I have a dataset with 4000 samples (50% positive and 50% negative). Normally I would do cross validation for this approach, however besides normal data mining techniques I am also ...
1
vote
1answer
21 views

Plotting Scatterplot Matrix or Correlation matrix or both?

I have a problem where I want to use a classifier for it. So I defined a set of features and created a dataset. Now I want to generate some plots to understand the features. I came across the ...
2
votes
0answers
18 views

Anonymizing survey data, plus an ethics question

This is a fairly simple question about raw data in Excel. I have a survey asking respondents to rate 17 questions on a rating scale from 1 to 5. The goal is for individuals who answer the survey to be ...
4
votes
2answers
109 views

Should types of data (nominal/ordinal/interval/ratio) really be considered types of variables?

So for instance here are the definitions that I get from standard text books Variable - characteristic of population or sample. ex. Price of a stock or grade on a test Data - actual ...
3
votes
2answers
64 views

What are some methods for generating simulated time series data for use in modeling?

I have a data set which consists of 50 observed years for which I have date and inflow values between a river and a reservoir. The data is formatted as follows: ...
2
votes
3answers
74 views

Gaps in time series and time series validity

After doing some reading on CrossValidated, I understood that we can use "imputation" techniques to fill in the gaps (if they are random). But I am not clear on following questions: How many ...
2
votes
2answers
71 views

How to get rid of bias in data?

I have been trying to classify a set of data into one of four classes. The data has already been generated and I have set aside 10,000 for training and 2,000 for testing. I have also generated the ...
0
votes
0answers
8 views

LETOR data sets, which contain the queries and documents?

While researching information retrieval evaluation, I found LETOR. This is a test collection used for evaluation of ranking algorithms. Since the data contains relevance judgements for queries, it can ...
0
votes
0answers
52 views

Training and testing on Unbalanced Data Set

I used SMOTE algorithm in R for class balancing. My data size has 13000 rows, I had 7% minority class in my sample now I used SMOTE( Synthetic Minority Oversampling Technique) for class balancing such ...
0
votes
1answer
60 views

Data Mining, generating a model from a database

I have a database with different variables which contains information such as age, date of vaccination, number of doses as well as the number of antibodies (which is my target variable). If the number ...
0
votes
0answers
61 views

Method for detecting anomalies in timeseries?

I am exploring creating alerts for anomalies in a timeseries. Say that I have a graph like this: There are two anomalies here that I would wanted to be alerted on: First, on April 7th with there ...
1
vote
1answer
54 views

Predictive analysis of data using R

I am trying to analyze data to predict the effect of 8 predictors on a response variable. I ant to do it using R. There are 8 predictor variables (x1, x2, ..., x8) and one response variable (y). I ...
2
votes
1answer
58 views

Interaction effects in big data sets

I'm looking for a method to identify a shortlist of potentially good 2-way interaction terms rather than trying all possible interactions. This question is similarly asked before here but in a more ...
0
votes
1answer
38 views

Data preparation and machine algo for ad click prediction

I am an ml noob. I have a task at hand of predicting click probability given user information like city, state, os version, os family, device, browser family browser version, city, etc. I have been ...
0
votes
0answers
11 views

Data Mining technique for preferred vendor problem?

Currently I use open auction method for selecting a transporter and want to switch to preferred transporter where I would select a transporter based on my preferred transporter list. The project ...
0
votes
0answers
12 views

Dataset for classifying image data, containing spatial info

I am looking for an object classification data set that meets some criteria: It should already have high-level local features extracted from the images (ideally taken from a grid) (less than 100 ...