**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

-1
votes
0answers
6 views

Merge and transposition of datasets [on hold]

I am having problem with merging 4 datasets into R from xlsx format. I want to create a dataset from 4 different xlsx files for specific observations. Lets say row ...
1
vote
0answers
17 views

Quantifying comparison of two different field data sets for the same 3D domain

I have a 3D domain (say a cube). The cube is discretized into an array of grid points. Each grid point holds a field data (like temperature, temperature gradient etc.). I would like to see if the ...
0
votes
1answer
38 views

Is there a database for non time series data sets? [duplicate]

I am searching for some data sets that would be suitable for regression analysis that my teacher wants for Statistical Learning lesson's homework. But as I search I only encounter with time series ...
0
votes
1answer
15 views

Determining two most similar objects based on multiple variables

I am trying to determine the two most similar years based on a number of environmentally variables. For example, I would like to choose the most similar year to 2015, from the set 1989 to 2016, based ...
0
votes
0answers
8 views

Test for determining likelihood of finding disease in 1 of 2 lymph nodes in patients with cancer

I am performing a study looking at two specific lymph nodes in the head and I want to be able to say if disease is more likely to be present in 1 or the other lymph node. Also, if disease is present ...
0
votes
0answers
12 views

Uncertainty on the standard deviation of data set, when the data points are uncertain

I have a set of four data points (x,y), and I would like to describe the spread of the data about the mean in both the x-direction and in the y-direction. Each individual data point has an uncertainty ...
0
votes
0answers
10 views

Is using pairwise comparison instead of single label value a valid way to augment training data

There are many papers using siamese architecture to do pairwise ranking. For example: suppose my training set is $\{X,Y\}$, where $Y$ is the continuous value. I can find a regressor to minimize $\|h(X)...
0
votes
0answers
11 views

Machine learning implementation in Ultrasonic sensor data [on hold]

I am using an ultrasonic sensor to detect liquid level(milk) in a container(silo). Length varies to 25 cm to 8 m. I am looking for ideas on machine learning implementation of the data received. I ...
0
votes
0answers
14 views

How to prepare data for anova or manova

I have a datafile (below is Gdrive link) and I want to prepare it for Anova/Manova tests. About Data: ...
0
votes
0answers
12 views

How to convert a Scikit-learn dataset to a Pandas dataset? [migrated]

How do I convert data from a Scikit-learn Bunch object to a Pandas DataFrame? ...
1
vote
0answers
14 views

Statistical Approaches for Uncertain Target Variable

Let me first explain what I have tried to mean by uncertain target variable. Consider the following scenario: Illegal immigrant behavior: Suppose you want to interdict with illegal immigrants to ...
2
votes
1answer
25 views

estimating variance using only data at the tails without resorting to Gibbs sampling

Suppose we know that the population size is $n=1,000$ but for whatever reason, we only have the bottom $n_1=100$ observations and the top $n_2 = 200$ observations. Furthermore, suppose we know the ...
0
votes
1answer
23 views

Data visualization for segments in a grid?

I'm working on a segmentation of individuals who have particular purchase behaviors. The net result is a set of 20 segments of people based on their consumption levels across two axes. An example ...
0
votes
0answers
13 views

How can I write an R script to check for straight-lining; i.e., whether, for any given row, all values in a set of columns have the same value [migrated]

I would like to create a dichotomous variable that tells me whether a participant gave the same response to each of 10 questions. Each row is a participant and I want to write a simple script to ...
2
votes
0answers
17 views

Impact of inverting grayscale values on mnist dataset

http://yann.lecun.com/exdb/mnist/ Pixel values are 0 to 255. 0 means background (white), 255 means foreground (black). Is there a reason why the original ...
1
vote
1answer
89 views

Data Comparison in R [closed]

I have to compare two data sets to make sure that they are identical. They are massive data sets with millions of data points. Is this possible in R and if so how would I go about that? I am brand new ...
0
votes
2answers
74 views

Is log transforming square root transformed data a legitimate data transformation?

Is it legitimate to do a "double transformation" on data? Specifically, log transforming data which has already been square root transformed, or conversely, square root transforming log transformed ...
0
votes
0answers
47 views

What is the most effective way to get a baseline rating prediction from netflix rating data?

I am doing an assignment where I am working with some of the Netflix Challenge data. I have a database that maps movie ids to customer ratings of that movie and a database of customers mapped to ...
1
vote
0answers
35 views

build my own 'pickle' image dataset for CNN training

I am trying to use Python & Tensorflow to make Convolution Neural Network to classify images. I found some examples from the Udacity assignment4 for notMINST image set. Now my question is : I ...
0
votes
0answers
22 views

Using polychoric correlation with non normal ordinal data

I have ordinal data on scale 1-5 for detected pollutants in water (1 = detectable in small proportions; 5= detectable in higher proportions; also 0 was asaigned - not detectable). based on the ...
0
votes
0answers
8 views

Separate duplicates of a data.frame into another data.frame [migrated]

I have a data.frame x, from which I want to put all duplicated rows in a new data frame z. My data frame looks like this: ...
1
vote
0answers
17 views

Creating a blended dataset/transitioning from one predictive model to another

I'm working with two different models right now using a variable X as an input. One of the models is said to be applicable for values of X less than A, and one is said to be applicable for values of X ...
1
vote
0answers
23 views

Prediction with large inputs and outputs

Being a total newbie with machine learning, I thought I would seek your advice on a problem of mine. I'm looking for any leads, starting points ans help you could provide me. I'm looking to predict ...
1
vote
0answers
16 views

Labeling methylated/unmethylated sequences [closed]

I'm trying to do some test with matlab/SVM to classify methylated/unmethylated human sequences. I'm using these sequences freely avaible at this address: http://services.ibc.uni-stuttgart.de/name21/...
1
vote
0answers
21 views

How to convert standardized line graphs into a function?

I have graphed revenues from 4 different products of the same variety (ie. four different movies and the revenues they brought in). The axes are: x is the weeks and y is the revenues; with the lines ...
1
vote
1answer
66 views

How to summarize data set in less than 100 bytes?

I'm in the beginning of a process learning to using accelerometer data to measure how good/bad a person drives a car (driver's behaviour). I want to decompose the acceleration into breaking, forward ...
0
votes
0answers
27 views

Decision tree implementation using ID3 algorithm (C# using AFrog.Net)

I am using ID3 algorithm to implement a decision tree and find an output for a given input query. For this I have a past data set and I am trying to implement the application as this code But ...
0
votes
0answers
11 views

How to name this type of Meter Reading Data?

I have some time series data that takes reads whenever a change in status happens. Reads are not recorded at any other point except for the state change. The reads are boolean values as otherwise this ...
1
vote
0answers
19 views

Do I Need to Normalize Data and Compensate for Apparent Overdispersion in Funnel Plots Generated from a 1.2 Million Record Dataset?

I have an anonymized dataset with 1.2 million dog and cat surgical sterilization records from 81 clinics. I would like to measure inter-clinic variation in mortality rate as part of a preliminary ...
1
vote
2answers
90 views

What is the statistical method that I should use for the following problem?

I have been given a problem and need to find the best statistical method that would help. The problem is as follows: A business has approx. 100,000 active customers who spend between £500 per annum ...
0
votes
0answers
47 views

Comparability of the data from two samples

I have a data sample with 600 units, this data sample is representative. From these 600 units I compute economic sentiment indicator on monthly base in period 2006-2010. From 2011 I want to change (...
0
votes
0answers
59 views

Data normalization/standarization and comparing

I have a question related to data comparing. First of all, my dataset is composed by cities of the world. In this cities, we have a maximum of 24 tags that indicate what these cities are best for. ...
2
votes
0answers
12 views

Are we lacking interesting Creative Common datasets? [closed]

I and a group of fellows are currently self-teaching some statistical methods using online resources. We mainly use R to tinker with algorithms such as K-means clustering and linear regression. ...
0
votes
3answers
32 views

how to predict sales of an item changes based on a discount given to another item?

I am developing a system where the management of the supermarket can make decisions on past sales data. There I mainly want to focus on how to predict sales of an item changes based on a discount ...
0
votes
1answer
24 views
0
votes
0answers
6 views

How to segment hours of audio for speech recognition?

I have 36 hours of speech data along with transcription. I'm planning to have 7 second audio segments, because I don't know any better. Suggestions are welcome. These segments will be passed through ...
0
votes
0answers
9 views

Suggestions for finding free Canada data sources [migrated]

before doing data analysis, I have to find public data sources first. The goal is to find housing services related data, or people data related to these housing services, in different regions of Metro ...
0
votes
0answers
14 views

Right & Left data censoring

I have question regarding Data Censoring Methods. Is there any link where I can find details about Left and Right Censoring? Also does anyone have any clue of Articles where someone tried to merge ...
0
votes
1answer
17 views

simulate sub group data for logistic regression

I want to simulate a data set for logistic regression in which my $Y_i \sim Bin(n_i, p_i)$ and $n_i >1 ~ \forall i$. I want something like: In another question, data has been generated for a ...
0
votes
1answer
17 views

How to address edge relations between training set and test set?

Suppose we are working on some sort of classification problem, and we have subdivided our data into a training set and a test set (or validation set, or etc.). We wish to prepare the data in the ...
3
votes
2answers
33 views

Relationship between categorical factors

I am not sure what this is called in English, but if we have two categorical factors, we can say that one of them (A) is finer than the other (B) if it holds true that if two observations belong to ...
0
votes
0answers
50 views

Data Aggregation Using ICC(1) & ICC(2) & Rwg

I am currently working on a thesis in order to graduate, and I need help to understand how to justify aggregation. My issue is that I don't quite understand what is require to do to test ICC(1), ICC(...
1
vote
1answer
20 views

How to evaluate level of significance of two similar correlations?

I have four datasets: A1, A2, B1, B2. Every dataset has between 100-300 items. Every item in every dataset has two values: x, y The goal: Find what datasets have similar x values. If the datasets ...
0
votes
0answers
7 views

to constrain a parameter estimate to obtain unique estimates in R LAVAAN for CFA

I need urgent help, I need the script or command to constrain a parameter estimate to obtain unique estimates in R LAVAAN for CFA. Like If Social Influence got 4 items in a model and I want to put a ...
0
votes
1answer
47 views

Level of factor taken as intercept?

I am using a GLM to analyze binomial data from one factor (Group) with three levels: Control, Control Treatment and Treatment. ...
2
votes
0answers
33 views

Dealing with noisy/mislabelled dataset

I have several datasets where each instance has numeric label assigned by a human that can take values between 1 and 5. After doing a manual inspection of one of these datasets, I noticed the ...
1
vote
0answers
20 views

What is the most popular file type for datasets? Why smile don't have database interface? [closed]

I just read smile document and it support many type of dataset file, but I want to know which dataset file type I should to stick around and why do they don't have database interface on smile? In the ...
0
votes
0answers
28 views

How to predict link using rooted pagerank

I am studying link prediction in social networks and I am trying to implement algorithms like common neighbors. I can't understand rooted Pagerank; it's an algorithm who calculate the similarity ...
1
vote
0answers
79 views

How to model a 2x2x2 dataset for a 3 way repeated measures ANOVA analysis

I have some data derived from a 2x2x2 experimental design. There are 3 IV's, each with 2 levels. My DV is reaction time (measured in seconds). I've used a repeated measures design (n=10). I have 20 ...
1
vote
0answers
14 views

Tests for ranked nominal data

I badly need help with the following problems. In my questionaire I asked respondents to name the three most important priorities (in order of importance) in connection with a specific area of life. ...