Tagged Questions

Structured data files in any format, collected together with the documentation that explains their production or use.

learn more… | top users | synonyms (1)

0
votes
1answer
19 views

Similarity datasets

Given two (or maybe more) datasets with the same samples/members, but with different variables. Is there a general way to compare the information available in the two datasets without looking into the ...
0
votes
0answers
12 views

I have set of data, I would like to understand the threshold for deciding good,average and poor [on hold]

I have a data of different visits of individual products to my website, I would like to segregate these based on a threshold. Currently there is a threshold set for good traffic, bad traffic and ...
0
votes
0answers
3 views

making for making dataset for spam classifier

I'm trying to make a dataset for spam detection, what is the best automatic selection that I can use for picking up the word that should i count for my spam base ? Thank you
0
votes
0answers
12 views

Missing Values in Dependent Variable and the Question on Tobit model (Panel Data)

I would appreciate some help and guidance on this issue. I have a panel data that looks like this. What I want to do is use the FDIflow variable as a dependent ...
0
votes
0answers
15 views

Medical Malpractice Datasets [on hold]

I hope this is the right place to ask this question. I'm looking for medical malpractice and error datasets from hospitals. To be very precise, for a given hospital I'd like a total count of deaths ...
0
votes
0answers
14 views

Multi-file data Cleanse Import How To [closed]

I have 1500 fields with the same columns and headers in .xls form. What do I use to mass clean these files before importing into our system? Would I do a batch import into Access? If so how? Would I ...
0
votes
0answers
12 views

How to merge dating correctly [migrated]

I'm trying to merge 7 complete data frames into one great wide data frame. I figured i have to do this stepwise and merge 2 frames into 1 and then that frame into another so forth until all 7 original ...
-1
votes
0answers
12 views

Need a large “interesting” dataset to illustrate LASSO [closed]

Does anyone know some interesting, publicly available, dataset for the sake of illustrating LASSO? It should, preferably, contain a lot of predictors ("large p"). I'd like to use it in my class.
1
vote
0answers
22 views

Adding real zeros to a dataset vs. presence-only modeling?

I have a fisheries dataset for which I have calculated the number of fishing sets in each grid cell (100 km x 100 km) for each month of every year. Fishermen in this fishery are legally required to ...
0
votes
0answers
10 views

Unique function leaves out column [migrated]

Say I use the UNIQUE function of R to create a script to pull out particular columns of a premade dataframe to make a new one: ...
2
votes
1answer
32 views

Publicly available data [closed]

I have some research data (around 300 GB) I want to make publicly available for other researchers. Do you know where can I upload these data?
0
votes
0answers
9 views

Training set wireless forensics tool

I'm working on a wireless forensics tool, starting with simple Dos attack ( de-authentication, disassociation, etc... ) before I start the very exhausting task of making my own training set, I thought ...
1
vote
1answer
31 views

Choosing the correct statistical test

I have two datasets that describe reports of caregivers and parents from 8 daycares. The two data sets are matched by daycare's name. We want to merge the data into a single dataset that will contain ...
2
votes
1answer
74 views

How can I quantify the difference between two categorizations/distributions?

I have a sample of survey data comparing two populations, non-addicts (sample size about 50000) and addicts (sample size about 500). The differences between mental health evaluations of the two ...
0
votes
0answers
21 views

Dataset with mixed structured/unstructured data

I am looking for some mixed-type dataset with free-form textual features along with some structural features (preferably continuous) - labeled, for supervised setting (classification or prediction). ...
0
votes
1answer
28 views

Temporal linkage between two lists of data

I have 2 lists, representing a time series: list_a = [23,43,29,45,6,12,240] list_b = [13,23,11,35,60,52,40] i.e. list_a[0] is ...
1
vote
0answers
17 views

How do I read in columns from a file with R [migrated]

I have files of the following format (just numbers, nothing else): ...
0
votes
0answers
21 views

Sufficient data volume for inference

This question was probably discussed before but I failed to find it here. Let's say I have a medical data which contain some biomarker measurements for cases (patients with disease) and controls ...
0
votes
0answers
20 views

Filling in missing values in one dataset based on another in presence of repeated observations in R [migrated]

Using R, I would like to use information from dataframe 2 to fill in missing values in dataframe 1. Here are the headers from my files. File 1 is a dataframe with data and location (long/lat) of an ...
1
vote
1answer
24 views

Benchmark datasets for testing multiple regression or multivariate regression model?

I have a question as a newbie. I'm working on a tool using regression analysis( linear, multiple, multivariate) to derive a regression model. To verify the correctness of the tool, I'm trying to find ...
1
vote
0answers
13 views

Is data fusion suitable for this application?

I have a situation where the delay vehicles experience at an intersection can be obtained through 3 or 4 types of sensors. Each sensor provides its own type of data with their own unique ...
0
votes
0answers
26 views

What is my chart type called?

I have my main data feed which only collects a new value if it is different from the last (it checks every 13 seconds) and from this I build a line chart. Is there a name for this type of chart? ...
0
votes
0answers
15 views

Null Values in Survey Data

What is the recommended approach for cleaning/preprocessing survey data that contains null values for some responses? An example of some responses are as follows: "Were you treated with respect?" ...
0
votes
0answers
18 views

algorithm to predict cost function

The goal of problem is to predict the weight for missing data . I have a dataset of categorical type as shown below ...
0
votes
0answers
20 views

How to merge overlapping discontinued data time series?

I have two datasets consisting of the US Federal debt held by Federal banks with a timespan covering the period 1953Q1 to 1988Q4 and 1970Q1 to 2014Q2. (series FDHBFRB and FDHBFRBN from the FRED ...
1
vote
1answer
45 views

Can you identify what the sharp spikes in our data could be?

Scenario: There was an online competition to win a prize. The competitors are technology start-up companies and the prize is worth around 5,000 USD. There was a public, online vote to decide the ...
1
vote
1answer
33 views

Publications in the social sciences or psychology based on freely available datasets?

There are a lot of datasets freely available across the web (see CV questions here, here, or here). Yet, I want the reverse: I am looking for a list of publications in the social sciences or ...
4
votes
1answer
61 views

How do I cite a dataset in a paper?

I want to use the iris dataset provided by scikit-learn for a paper. But I don't know what the standard for referencing datasets is. What citation should I use for this dataset in my paper? Should I ...
0
votes
0answers
24 views

Have a data set for 3 consecutive days. What are my options?

Let's say I have a set regarding the transportation methods(Eg: car, bus, train) used for three consecutive days b y $n$ number of people. For simplicity let us assume that everyone use only one type ...
0
votes
1answer
30 views

Why does my repeated measures ANOVA analysis vary when I organize my data differently?

In my study there were 63 types of stimuli (63 levels of the IV) and all types had 8 different examples (8 trials per level of IV) which were all categorized by each participant. Accuracy of ...
0
votes
0answers
8 views

Multiple combinations (ex drug-ADR) with the same unique case ID.

I am quite new to R statistics, and I one you can help me. I have tried finding the answer to my question by searching the forum and so on, and I apologize in advance if my question is trivial or ...
0
votes
0answers
36 views

Data structure for Inverse Cumulative Distribution when columns are *NOT* separate variables

I am trying to structure data to conduct a Frequency analysis and output an Inverse Cumulative Distribution histogram for a biomechanics study. The goal is to compare 4 devices with respect to their ...
0
votes
0answers
37 views

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced good results?

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced state-of-the-art results? Even if its not state-of-the-art, but, I am looking for ...
1
vote
1answer
11 views

What method should I use to model simplified performance

I have the data of some build times. These builds occur on a cloud and the runtime has been more varied than in the past. One thing that could be occurring is an improper sharing of resources ...
0
votes
1answer
46 views

Once you have a set of training data, how do you go about training a model?

I've been trying to teach myself machine learning using some books, kaggle and other resources so I'm still very new at this. I'm trying to do a competition on kaggle and I've obtained a dataset. ...
0
votes
1answer
33 views

Standard model for time series with (possibly multiple) seasonal component

Suppose you have a "new" way to formalize a seasonal component and you want to see if your method is worth to be published. My idea is to take a "standard" model for time series with seasonal ...
0
votes
0answers
14 views

Mathematics of simple performance testing [duplicate]

I have a set of sorted tables T that have known but different dimensions. There are two types of functions in this system: f(T) g(T, n), where n is an integer parameter. ... and two types of costs ...
0
votes
0answers
6 views

Data Collection Design

Would like some critique on data selection for testing. I am planning some indoor imagery localization experiment. I have two kinds of data (i) Image coming to me at 25HZ (ii) telemetry coming to me ...
1
vote
2answers
39 views

Large data set, need PROPER way to calculate averages

I'm working on an excel sheet with 26k lines and need to calculate average costs per KG. I'm not sure which method of calculation to use as each is giving me different results. For example: ...
1
vote
1answer
36 views

Missing data analysis software

Does any standard statistical software like R, SAS or SPSS have procedures or codes to analyze log-linear models for missing data in contingency tables using maximum likelihood estimation (or EM ...
1
vote
1answer
17 views

Supervised learning algorithm that can be easily retrained with new data

I have a web crawler and i want to be able to differentiate a specific class of website (social networks), from others. My problem is that my starting classified data is really small. What I wan't to ...
0
votes
0answers
23 views

when to use panel time-series regression vs seeming unrelated regression

I am a bit confused on whether or not I have to use a fixed-effect panel time-series method or SUR (seemingly unrelated regression). To get a background of what I am trying to do, I have 10 panels of ...
1
vote
0answers
10 views

Estimate average of a dataset when thresholded

Suppose I have a dataset $x_i$ (with $x_i >= 0$) of N samples. I would like to determine the average of the data when the data is limited to some theshold c: $limitedaverage(x,c) = \frac{1}{N} ...
0
votes
0answers
25 views

Normalize data in unnormalized data after normalization

i have some data in numeric. I want to do some classifications method. So i decided to check the normalization of the data. I have normalize my data using weka tools. And i think weka had normalized ...
6
votes
1answer
139 views

What are some interesting examples of wrong or crazy inferences being drawn from Big Data?

I'm interested in well-known examples of Big Data misinterpreted, poorly analysed, or wrongly employed to unscientific and incorrect ends. Would appreciate any examples or observations. Thanks a lot!
3
votes
1answer
79 views

Missing data which simply cannot exist

I have read 80% of missing data in a single variable and understand the approach for dealing with missing data which simply cannot exist for 1 variable. I am trying to generalise this up to 2 or ...
0
votes
0answers
23 views

Consistent Questionnaire Responses

I want to ask please questionnaire responses in Likert Scale. I have got some respondents who had consistent answers for a large number of questions. For example, over 75% or 80% answers (in different ...
8
votes
2answers
145 views

Analysis of hamster wheel rotational data

Before posting this question, I had a browse through other questions on this Stack, and this is probably going to be a walk in the park for anyone who reads it! But the description of this Stack ...
1
vote
1answer
58 views

How to determine whether a set of data are qualitative or quantitative?

Assume a data set with multiple columns, where the categorical data are coded. What is the best rule(s) or rule of thumb to determine whether each column contains qualitative data or quantitative ...
0
votes
1answer
30 views

Data imputation for statistical analysis

Here is the situation: I have an individual level data set $X$ where each row is a person $i$ and each column denote characteristics of $i$. The problem is that my data is missing an important ...