**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

0
votes
0answers
9 views

Scraping data from web application? [on hold]

I am trying to get data that is displayed from what seems to be a javascript app (not sure). For example, consider this page. The graph under the "Finance" tab contains pricing information about some ...
0
votes
1answer
8 views

How to address edge relations between training set and test set?

Suppose we are working on some sort of classification problem, and we have subdivided our data into a training set and a test set (or validation set, or etc.). We wish to prepare the data in the ...
0
votes
0answers
4 views

how to find principal vector and prencipal component from kernel matrix?

i have kernel matrix n*n it as gram matrix how i find U as principal vector as alpha and V as principal component PC by solving eigen value problem as lamda ?how i normalize them ?how i project the ...
0
votes
0answers
18 views

Copy a cell in excel every time it changes (tennis match) to get data automaticly [on hold]

I have a program that is connected to Excel. Which means there are cells that are constatly changing automaticlly (after every point). There are cells showing number of SETS,GAMES,POINTS of each ...
3
votes
2answers
27 views

Relationship between categorical factors

I am not sure what this is called in English, but if we have two categorical factors, we can say that one of them (A) is finer than the other (B) if it holds true that if two observations belong to ...
0
votes
0answers
15 views

Data Aggregation Using ICC(1) & ICC(2) & Rwg

I am currently working on a thesis in order to graduate, and I need help to understand how to justify aggregation. My issue is that I don't quite understand what is require to do to test ICC(1), ...
1
vote
1answer
16 views

How to evaluate level of significance of two similar correlations?

I have four datasets: A1, A2, B1, B2. Every dataset has between 100-300 items. Every item in every dataset has two values: x, y The goal: Find what datasets have similar x values. If the datasets ...
0
votes
0answers
4 views

to constrain a parameter estimate to obtain unique estimates in R LAVAAN for CFA

I need urgent help, I need the script or command to constrain a parameter estimate to obtain unique estimates in R LAVAAN for CFA. Like If Social Influence got 4 items in a model and I want to put a ...
0
votes
1answer
43 views

Level of factor taken as intercept?

I am using a GLM to analyze binomial data from one factor (Group) with three levels: Control, Control Treatment and Treatment. ...
2
votes
0answers
31 views

Dealing with noisy/mislabelled dataset

I have several datasets where each instance has numeric label assigned by a human that can take values between 1 and 5. After doing a manual inspection of one of these datasets, I noticed the ...
0
votes
0answers
6 views

Cross validation of dataset separated on files [migrated]

The dataset that I have is separated on different files grouped on samples that know each other, i.e., they were created on similar conditions on a similar time. The balance of the train-test dataset ...
1
vote
0answers
19 views

What is the most popular file type for datasets? Why smile don't have database interface? [closed]

I just read smile document and it support many type of dataset file, but I want to know which dataset file type I should to stick around and why do they don't have database interface on smile? In the ...
0
votes
0answers
14 views

Need help with proper entering my data into spss [closed]

My research was about improvements in tapping a circle with or without feedback. We had 31 people tap a moving ball about 320 times. We measured the deviation from the ball and we want to see if there ...
0
votes
0answers
20 views

How to predict link using rooted pagerank

I am studying link prediction in social networks and I am trying to implement algorithms like common neighbors. I can't understand rooted Pagerank; it's an algorithm who calculate the similarity ...
0
votes
0answers
13 views

Age, Weight and Height dataset [migrated]

I am trying to get a dataset (preferable worldwide) that has the attributes (height, weight and age). The only decent dataset that I have been able to find was from here: Human body organs growth ...
1
vote
0answers
65 views

How to model a 2x2x2 dataset for a 3 way repeated measures ANOVA analysis

I have some data derived from a 2x2x2 experimental design. There are 3 IV's, each with 2 levels. My DV is reaction time (measured in seconds). I've used a repeated measures design (n=10). I have 20 ...
1
vote
0answers
13 views

Tests for ranked nominal data

I badly need help with the following problems. In my questionaire I asked respondents to name the three most important priorities (in order of importance) in connection with a specific area of life. ...
0
votes
1answer
25 views

type of image to create a dataset for image recognition using convolution neural network

I was trying to create a dataset for animal detection using convolution neural network. It was for some open source project. For the training and testing, I thought to create a dataset myself. for ...
0
votes
0answers
12 views

Validating log-log fit

I'm trying to build a predictive model with my data. I selected the log-log transformation with the idea that it could tame the heteroskedasticity in the data set and it appeared to do so well. ...
0
votes
2answers
52 views

How do I remove outliers in dataset?

I have a data-set (185 rows) with 20 predictors and 1 dependent variable. I have applied Cook's distance and then 4/N formula to remove some of the outliers in 1st iteration. Should I do this ...
0
votes
0answers
19 views

How to assess the separable capability of a dataset

I'm using several a convolutional neural network to classify my datasets. In some datasets, I obtain very good accuracy (~80%). These datasets have different number of examples, from 10,000 to ...
0
votes
0answers
16 views

Combining independent observables for the same quantity, how to check independence

Let's assume we have a set of observations $\{x_i\}$ of a physical quantity $x$. We have a model $x=X(y)$, where $y$ is a free parameter. Let's assume that we have two distinct observables, $f(x)$ and ...
0
votes
0answers
14 views

Proportional data - Which test

I've recently conducted an in class experiment on cockroach behaviour. But I'm stuck on how to analyse the data. We aimed to seek if the cockroaches would spend more time at the edge of our ...
0
votes
0answers
14 views

How to aggregate many simulation runs for curve-fitting?

I have results from about 200 runs of a simulation model. The results contain a stochastic response variable which I want to approximate with a curve-fitting approach. So far, I have opted for ...
1
vote
0answers
8 views

pooling data of mice from 2 independent experiment

I am measuring inflammatory parameters in animals where i have 4 groups. Wild type mice treated/untreated and Knockout mice treated/untreated. the aim is to see whether knockout mice have a ...
0
votes
0answers
3 views

Storing hierarchical data as a list of matrices in R [migrated]

Noob R question here from a Matlab/Python user. I have a dataset with hundreds of different users, each of whom has a unique number of rows of data, and would like to store the data as a list of ...
0
votes
0answers
7 views

Dataset dimension with ExtraTrees

We are using an ExtraTrees with Fitted Q-Iteration, and testing it with different size of dataset, all generated in the same way. FQI seems to have very good performance with small dataset, but it get ...
3
votes
0answers
14 views

How to deal with implausible answers to filter questions? [duplicate]

I work with a questionnaire that contains a variety of filter questions. In the following I will give an example how one block of this successive questions look like. Have you ever skipped ...
1
vote
0answers
27 views

Which CDF to use? [closed]

Hey guys I'm preparing a Statistics assignment and I have a very large data set showing the ground temperature over the years. Data set consists of temperature and temparature uncertainty. Example is ...
0
votes
0answers
7 views

How to get monthly Treasury Bill time-series data from weekly quotes

I am trying to get monthly T-Bill rate data for Sri-Lanka, for 3 and 6 month maturities. The Central Bank website has a downloadable excel spreadsheet giving weekly quotes (thus 4 rates quoted per ...
0
votes
1answer
22 views

Statistical analysis for replication in very small experimental datasets

A colleague does replication on a quite coslty experiments. There are four different conditions, each one duplicated. The outcome with $4\times 2 = 8$ points is illustrated below: The analysis is ...
0
votes
1answer
38 views

Finding outliers in multiple dimensions

I'm working on dataset which isn't normally distributed. It contains three dimensions: cost, discount and profit. I'm trying to find outliers in all these dimensions. I used $\text{z-score}$ to find ...
1
vote
0answers
12 views

Guessing/extracting from the pool of partially right guesses

Lets assume that I have integer numbers from 1 to 50. I imagine any 5 unique numbers from 1 to 50. A computer can generate an array of 5 random numbers, as many times as it wants. Each time it ...
1
vote
0answers
4 views

Open datasets for educational use [migrated]

I'm interested in compiling open datasets for educational use. For example, this page from the State University of New York Geneseo includes open datasets by topic. I'm especially interesting in ...
0
votes
0answers
5 views

achievable accuracy in common datasets

I have been using the credit approval dataset https://archive.ics.uci.edu/ml/datasets/Credit+Approval and I have achieved an accuracy of about 86%. I want to know how good this is. Is there a ...
0
votes
0answers
15 views

Reporting Data with Estimates and SE

What do you think? When reporting data in an article, a usual way is to show the mean values +/- the standard errors in a table. For Example, when you measure Abundance of pigeons in parks of five ...
1
vote
1answer
53 views

Probability that the age is a multiple of 10

I am working on a data set, containing demographic informations in a developing country. I noticed that a third of the ages is a multiple of 10 ! I expected (though the distribution of ages is not ...
1
vote
0answers
26 views

Looking for “Data gathering is hard and expensive”

I've finished some data mining and machine learning courses and in most of them at the very beginning has been said: "Collecting and pre-processing data is the most expensive and hardest part of the ...
0
votes
0answers
16 views

Useful inputs from files for neural network?

I'm building a visual neural network sandbox. ATM i'm working on the Input nodes, where you have a browser to choose a file from your drives and drop it in the workspace. The workspace detects what ...
0
votes
0answers
23 views

transformation in logistic regression model

So we know we can use BoxCox or inverse transformation in linear regression or continues dependent variable models. you can use boxcox() invResPlot() or powerTransform() in R for transformation your ...
0
votes
0answers
12 views

Topics needed to learn to for predicting future of a data set?

I am planning to work this summer by writing a code which will eventually become an app, that has the ability to predict what the prices will be atleast 6 month ahead of time. I have a data set of the ...
1
vote
0answers
36 views

Is the dataset Normally distributed?

So based from this link http://www.simafore.com/blog/bid/107702/2-ways-of-using-Naive-Bayes-classification-for-numeric-attributes I began to realize it might be a good idea to compute the pdf of ...
0
votes
0answers
14 views

regarding generating synthetic data simulating the real data

We are trying to develop some predictive models. The current scenario is that we have to rely on synthetic data at first since the real data set will not be available quite soon. It is understandable ...
1
vote
0answers
13 views

Multi-target regression Datasets [closed]

I'm looking for a dataset for multi-target regression . I found severals but these datasets have few instances and I need datasets with more than 100k instances. Does anyone know where I can find ...
1
vote
0answers
19 views

Which model(s) are appropriate for this kind of data

So, I tried to implement a model on some data. The dependent variable is a ratio that can get higher than 1, is lower bounded by zero and, seeing figure 1, is left skewed.Thus, a logit regression is ...
0
votes
0answers
19 views

Numerical Similarity Measures for Dirty Data

I need to compare two datasets to see if they are "similar" in terms of numerical similarity between given columns. The problem is, however, that I do not know the characteristics of the 2nd dataset. ...
0
votes
1answer
127 views

How to graph throughput

UPDATE 2: maybe a simple way to state it: for web application performance data consisting on: start timestamp | end timestamp | response time I'd like to compute a non-aggregated data set consisting ...
0
votes
0answers
8 views

Creating subsets of data with a particular target distribution

Is it possible to select a subset of observations from a dataset, while enforcing particular distributions across the data dimensions? In essence, create subsets of data with particular target ...
0
votes
1answer
13 views

Absolute Average deviation in percentage calculation

Sorry if my terminology is incorrect. I am trying to calculate the average error of prediction to be represented in percentage. For example, I should be able to say, the predicted values are on ...