Structured data files in any format, collected together with the documentation that explains their production or use.

learn more… | top users | synonyms (1)

0
votes
0answers
10 views

Are there free APIs for searching news articles that I can use to collect trend data in news coverage? [migrated]

I am working on a data-visualization web application that looks at trends in American news media coverage over time. It takes a keyword (or keywords), a date range, and a time increment as parameters ...
0
votes
0answers
4 views

Multiple combinations (ex drug-ADR) with the same unique case ID.

I am quite new to R statistics, and I one you can help me. I have tried finding the answer to my question by searching the forum and so on, and I apologize in advance if my question is trivial or ...
0
votes
0answers
9 views

Time imputation from interval censored-data for logistic regression

I have 200 individuals with a time $T_i$ (unknown) of infection that is included in the interval $[L_i,U_i]$ (data known) different for each individual. I suppose that Ti follows a lognormal ...
0
votes
0answers
28 views

Data structure for Inverse Cumulative Distribution when columns are *NOT* separate variables

I am trying to structure data to conduct a Frequency analysis and output an Inverse Cumulative Distribution histogram for a biomechanics study. The goal is to compare 4 devices with respect to their ...
0
votes
0answers
14 views

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced good results?

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced state-of-the-art results? Even if its not state-of-the-art, but, I am looking for ...
1
vote
1answer
10 views

What method should I use to model simplified performance

I have the data of some build times. These builds occur on a cloud and the runtime has been more varied than in the past. One thing that could be occurring is an improper sharing of resources ...
0
votes
1answer
40 views

Once you have a set of training data, how do you go about training a model?

I've been trying to teach myself machine learning using some books, kaggle and other resources so I'm still very new at this. I'm trying to do a competition on kaggle and I've obtained a dataset. ...
0
votes
0answers
21 views

Financial News Dataset [closed]

I'm looking for some financial news data set. I've already found Thompson Reuters Corpora (http://trec.nist.gov/data/reuters/reuters.html) but it's a little bit outdated, it has news only before 2005. ...
0
votes
0answers
6 views

Same data set but different method for analysis [migrated]

A data set has been analyzed in a research paper. Is it possible to use the same data set but different analysis technique and write a research paper?
0
votes
1answer
26 views

Standard model for time series with (possibly multiple) seasonal component

Suppose you have a "new" way to formalize a seasonal component and you want to see if your method is worth to be published. My idea is to take a "standard" model for time series with seasonal ...
0
votes
0answers
14 views

Mathematics of simple performance testing [duplicate]

I have a set of sorted tables T that have known but different dimensions. There are two types of functions in this system: f(T) g(T, n), where n is an integer parameter. ... and two types of costs ...
0
votes
0answers
6 views

Data Collection Design

Would like some critique on data selection for testing. I am planning some indoor imagery localization experiment. I have two kinds of data (i) Image coming to me at 25HZ (ii) telemetry coming to me ...
1
vote
2answers
37 views

Large data set, need PROPER way to calculate averages

I'm working on an excel sheet with 26k lines and need to calculate average costs per KG. I'm not sure which method of calculation to use as each is giving me different results. For example: ...
1
vote
1answer
32 views

Missing data analysis software

Does any standard statistical software like R, SAS or SPSS have procedures or codes to analyze log-linear models for missing data in contingency tables using maximum likelihood estimation (or EM ...
1
vote
1answer
14 views

Supervised learning algorithm that can be easily retrained with new data

I have a web crawler and i want to be able to differentiate a specific class of website (social networks), from others. My problem is that my starting classified data is really small. What I wan't to ...
0
votes
0answers
12 views

when to use panel time-series regression vs seeming unrelated regression

I am a bit confused on whether or not I have to use a fixed-effect panel time-series method or SUR (seemingly unrelated regression). To get a background of what I am trying to do, I have 10 panels of ...
1
vote
0answers
8 views

Estimate average of a dataset when thresholded

Suppose I have a dataset $x_i$ (with $x_i >= 0$) of N samples. I would like to determine the average of the data when the data is limited to some theshold c: $limitedaverage(x,c) = \frac{1}{N} ...
0
votes
0answers
20 views

Normalize data in unnormalized data after normalization

i have some data in numeric. I want to do some classifications method. So i decided to check the normalization of the data. I have normalize my data using weka tools. And i think weka had normalized ...
6
votes
1answer
118 views

What are some interesting examples of wrong or crazy inferences being drawn from Big Data?

I'm interested in well-known examples of Big Data misinterpreted, poorly analysed, or wrongly employed to unscientific and incorrect ends. Would appreciate any examples or observations. Thanks a lot!
3
votes
1answer
57 views

Missing data which simply cannot exist

I have read 80% of missing data in a single variable and understand the approach for dealing with missing data which simply cannot exist for 1 variable. I am trying to generalise this up to 2 or ...
0
votes
0answers
18 views

Consistent Questionnaire Responses

I want to ask please questionnaire responses in Likert Scale. I have got some respondents who had consistent answers for a large number of questions. For example, over 75% or 80% answers (in different ...
8
votes
2answers
130 views

Analysis of hamster wheel rotational data

Before posting this question, I had a browse through other questions on this Stack, and this is probably going to be a walk in the park for anyone who reads it! But the description of this Stack ...
1
vote
1answer
47 views

How to determine whether a set of data are qualitative or quantitative?

Assume a data set with multiple columns, where the categorical data are coded. What is the best rule(s) or rule of thumb to determine whether each column contains qualitative data or quantitative ...
0
votes
1answer
28 views

Data imputation for statistical analysis

Here is the situation: I have an individual level data set $X$ where each row is a person $i$ and each column denote characteristics of $i$. The problem is that my data is missing an important ...
0
votes
2answers
23 views

How to propositionalize a relational data set for clustering analysis?

I am working with a data set of students and their courses for a single semester, attempting to cluster based on the courses & various other attributes where "courses" are the "many" side of a ...
4
votes
2answers
165 views

Best way to represent data

What is the best way to represent the following data graphically? Can I use a histogram ...
0
votes
1answer
33 views

Do you know of any public circular/angular dataset? [closed]

I've been trying to locate and access circular/angular datasets for my research. I'm particularly interested in datasets that distribute according to a mean centered circular distribution with ...
0
votes
0answers
13 views

Term to Describe Strongly Clustered Data

I have some data which are strongly gathered into more than one cluster. I am looking for a term to effectively describe this phenomenon: e.g., multi-clustered data, which however seems to me that we ...
1
vote
1answer
41 views

statistical analysis with interval data

I have been assigned the task of helping someone who has collected some survey associated data related to some kind of government facilities provided to people. The data corresponding to small scale ...
0
votes
0answers
22 views

Categorical Variable with more than 50 levels

I am new to analysis. I would like to know relation between my 2 sets of categorical variables , Both my variables has more than 50 levels. Is it a good approach to create dummy variable and do the ...
-1
votes
2answers
41 views

How do you draw a random sample of unique IDs in a large dataset?

Rookie here -- I have a a large data set with about 75,000 observations, and 2000 unique IDs. Therefore, each unique ID has about 37 observations. I'm trying to draw a random sample of unique IDs, say ...
1
vote
1answer
23 views

Distribution comparison across different treatments

if I have a table like this In which, site = different sites of measurement (tissue or organ) type = different types of virus And I want to ask something about the data 1.What test can I use to ...
1
vote
2answers
37 views

clustering evaluation based on a gold standard

I have a dataset of items and I want to measure how my clustering method works. I'm using R and simple k-means clustering. I have a lets say gold standard of my clusters and I want to see how good is ...
0
votes
0answers
8 views

What is an equi-depth partition of the data?

In the paper Outlier Detection for High Dimensional Data at the beginning of section 1.3 Is written: Each attribute of the data is divided into $\phi$ equi-depth ranges. Thus, each range contains ...
0
votes
0answers
12 views

Telecommunication Billing DataSet

Has anyone seen any telco billing dataset available for analysis? I looked at Kaggle or http://www.d4d.orange.com/en/presentation/data but couldn't find anything.
0
votes
1answer
54 views

problems to build the linear multiple regression in R and to explain the output [closed]

I have a problem to build and to explain the linear multiple regression. I have a data set called Cars93 with 26 variables (numeric and not numeric) and 93 ...
1
vote
0answers
17 views

Longitudinal data format in WinBUGS or OpenBUGS, different number of items per time point

I am working with longitudinal data, where there are a different number of "items" per time point. I am working with the MCMCdynamicIRT1d function in ...
2
votes
2answers
31 views

Get the distribution of a dataset

this is a conceptual question. In data mining, the problem often arises that scientists/data engineers are using an expert guess for the underlying distribution of their data. Often their assumption ...
2
votes
0answers
20 views

Stores and ordered rank of products

I have data about popularity of some products in some stores. For each of the stores I have only a ordered rank of best-selling products in them. I need help how to find information about which ...
2
votes
2answers
36 views

Tool for generating correlated data sets

Does anyone know of a tool that I can use to generate a set of data with known correlations (and to put the icing on the cake - output this in json,csv,txt or some common format)? I am working on ...
0
votes
0answers
34 views

Using Cronbach's alpha to test reliability of a questionnaire

I administer a questionnaire to managers asking them to what extent they agree / disagree that they perform certain functions (listed in the questionnaire) on a scale of 1-5. The sample functions are ...
2
votes
0answers
42 views

Unbalanced Panel: pooled OLS vs FE vs RE - which method yield unbiased and robust estimators?

I am conducting an empirical study (relation between earnings and returns). I have an unbalanced panel with $N=449$ firms and $T=36$ time periods. Regarding the estimation methods I am confused: In ...
0
votes
0answers
33 views

compare strength algorithms across different data sets

Reflecting on this paper. The use of a statistical significance test is required in order to compare the algorithms across different data sets. In that paper and other papers, the number of data ...
0
votes
1answer
126 views

German credit data: neural network, svm, logistic regression : input variables

I'm using the following data set on some credit scoring models: https://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data) My teacher told me that it's best to use the same data set for all ...
0
votes
0answers
15 views

Removing labelling noise

I have a big data set with unlabelled observations (several million) and about 20 thousand properly labelled ones. There are only two classes and all correctly labelled samples belong to the same ...
1
vote
1answer
20 views

Bidding Collusion Question: Probability of Identical Bids

I was asked this question. My initial conclusion is that there isn't enough information to calculate the probability. I would appreciate it if anyone could provide provide their insight. Scenario: ...
0
votes
1answer
33 views

determine independent and dependent variables without apriori knowledge of the nature of data

i have a spreadsheet with seven parameters and their values for 9000 samples.I need to determine which are dependent parameters and which are independent. Please let me know which algorithm might be ...
1
vote
0answers
18 views

Regression of irregular time series

I am beginning to look at high frequency data sets. I have only ever dealt with regularly spaced data up until now so wondering what best practice is. If I have two time series X and Y with Y being ...
0
votes
1answer
36 views

Simulate data by using existing dataset

I have a complete dataset with input variables and response variables. I would like to perform a simulation where I give the input variables and generate randomly the response variables. Is there a ...
16
votes
4answers
287 views

Good data example needed with covariate affected by treatments

I have looked at a lot of R datasets, postings in DASL, and elsewhere, and am not finding very many good examples of interesting datasets illustrating analysis of covariance for experimental data. ...