**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

0
votes
0answers
9 views

Outliers detection methods for RNA-Seq data

I have a rna-seq dataset with normalization in RPKM. The dataset have 1 gene per row with 4 different experiment condition. I need a detect de outliers values in this dataset. I used de weka filter ...
1
vote
0answers
14 views

dataset size v.s. accuracy

I have a dataset of 100 users. (example) User1: 300 instances / User2: 500 instances / ... / User100: 200 instances where instance = {feature1, feature2, ..., featureN, class} Here, I computed an ...
0
votes
1answer
27 views

Statistical methods?

I have data from my experiment where two groups participated (monolinguals vs. bilinguals). There was a language switching as well as a task switching part to the experiment where the data tells how ...
-1
votes
0answers
17 views

Data science : where to find Project Examples using Python with SQL? [on hold]

I’ve learnt Python and SQL, but was suggested to “work in Python to analyze data sets stored in MySQL”. I am following CS109 which uses Python, but it does not really use SQL https://www.quora.com/...
-3
votes
0answers
14 views

Set Vertex Attribute by comparing ID Igraph [on hold]

I'm having difficulty merging a dataframe of vertex attributes with an igraph with vertex names and edges. My problem is that I want to merge the datasets by ID, but my vertices in the igraph and the ...
0
votes
1answer
31 views

The Causal Impact Package in R could not trace the right number in the test group

I have this sample dataset which is as follow: head(data) ...
1
vote
0answers
20 views

validating a gaussian process fitted to data

I am relatively new to applying Gaussian processes to data. I come from a math background but the most popular literature on it seems to be from a machine learning perspective and not from a ...
0
votes
0answers
7 views

The impact of the number of dimensions on classification/predictive performance

For the sake of simplicity, let's say we have a rectangular dataset where the columns are fields/variables and the rows are observations/events. According to the curse of dimensionality, the higher ...
0
votes
1answer
32 views

Proving that a dataset has all combinations of some features

What is the best way to prove / show that a dataset has all combinations of nominal values for a known group of features? For example, if attributes A and ...
-1
votes
1answer
11 views

datav alidation on lexcel [closed]

I have given data validation for Std. warehouse as "YGCTCAP1" and "YGCTBAP1" for the first row i entered YGCTCAP1 by drop down, and i drag down the cell, it in became YGCTCAP2, --3,--4 ETC , ...
0
votes
0answers
7 views

Where to find bulk public death records? [closed]

I want to do some statistical analysis and cross referencing with the set of all death certificates from 2010 onwards in the state of Pennsylvania. Where can I find a bulk download of public death ...
0
votes
0answers
19 views

How to simulate a dataset for linear modelling with a 3-level factor?

I am trying to simulate a dataset with a 3-level factor, and I hoped I could run some code by others to make sure I've not done anything wrong. The dataset is analysed in GLM with in R. The model ...
1
vote
1answer
17 views

How to rank ordinal data?

I have 33 indicators and I asked 16 experts to indicate the significance of each indicators using 5 points Likert questionnaire. So basically for each indicator I have 16 scores (each score varies ...
0
votes
0answers
17 views

Looking for GDP Data Per State [closed]

Where can I find historical GDP data for each state in the US? For example, I did some googling for chicago and only found data from 2001-2014. I'd like to find figures from at least 30 years back. ...
-1
votes
0answers
15 views

LIBSVM data format [closed]

New to LIBSVM, can someone tell me how I can format for example the ala datasets from the UCI collection in order to classify the data in Matlab.
0
votes
0answers
27 views

Association of qualitative and quantiative variables

I have data of an ordinal qualitative variable and I would like to study if it has some relation with a continuous quantitative variable and I don't know what technique could I use for that. I was ...
1
vote
0answers
24 views

hierarchical cluster analysis in SPSS with ordinal data

I have ordinal data on scale 1-5 for detected pollutants in water (1 = detectable in small proportions; 5= detectable in higher proportions; also 0 was asaigned - not detectable). I want to do HCA ...
0
votes
0answers
19 views

The effect of correlated sample(observation) on classification

I currently work on BCI Speller systems, and for my analysis, i have recorded EEG data from 8 different EEG channel.after preprocessing my data ,i have a 8*12*15*256 matrix.in general, researchers ...
0
votes
1answer
20 views

Comparison of 3 different data sets

I have two time varying datasets that are on different scales (one typically of the order of magnitude of 1e-3 and the other of order of magnitude 100). I've been working on a process that generates a ...
1
vote
2answers
19 views

How does k-fold cross validation fit in the context of training/validation/testing sets?

My main question is with regards trying to understand how k-fold cross-validation fits in the context of having training/validation/testing sets (if it fits at all in such context). Usually, people ...
2
votes
1answer
30 views

Which graph/network format should be used when releasing a data set?

Under RENOIR (an EU funded project) I am working on releasing the data set of Slovenian Press Agency for researchers and other folk. As part of the data set we will be releasing some graphs with ...
0
votes
1answer
19 views

How to compute logarithmic loss with the probability output of binary classification

The following code are used to produce the probability output of binary classification with Random Forest. ...
0
votes
0answers
13 views

Quantifying comparison of two or more 1D datasets (pdfs or cdfs)

I have two 1D data sets which represent the same field quantity and are of same size. Say A1 and A2 both are arrays of size 1 million . Now I have to quantify the comparison of these two data sets ...
0
votes
0answers
13 views

Calibrate grouped, unpaired data

I have a calibration dataset (A) and a dataset I want to calibrate (B). Each observation consist of a species name and a measured value. The data is not paired; an individual is either in A or in B, ...
0
votes
0answers
21 views

Synthetic data generating for dimension reduction algorithm testing

I would like to ask if somebody can help me with the following problem. I would like to generate synthetic data for dimension reduction algorithm testing. Specifically, I would like to have for ...
0
votes
0answers
3 views

Reading time series data into R when you don't have time labels? [migrated]

We have a data set that is in a very strange format. It mentions the start date and the end date and time and then goes into data values. See the format sample below: ...
0
votes
0answers
51 views

How can I prove that the data that I have can't answer the question being asked of it? [duplicate]

One possible successful outcome in a data science experiment would be learning that the data can't answer the question being asked of it. I was wondering if there are some tests that I can perform to ...
0
votes
0answers
5 views

Are SAS Data Storage Options designed for Big Data? [migrated]

I understand that SAS has some data storage options in the form of Default Storage tables or SAS datasets - in the SAS library. Multi-dimensional Storage cubes - in the OLAP Server. Third Party ...
0
votes
1answer
19 views

How to measure if data conforms to logarithmic curve

I am collecting data that should closely resembles a logarithmic curve. I have many datasets. How can I measure how closely each dataset resembles a logarithmic curve and call out any outlying ...
1
vote
0answers
22 views

Quantifying comparison of two different field data sets for the same 3D domain

I have a 3D domain (say a cube). The cube is discretized into an array of grid points. Each grid point holds a field data (like temperature, temperature gradient etc.). I would like to see if the ...
0
votes
1answer
41 views

Is there a database for non time series data sets? [duplicate]

I am searching for some data sets that would be suitable for regression analysis that my teacher wants for Statistical Learning lesson's homework. But as I search I only encounter with time series ...
0
votes
1answer
20 views

Determining two most similar objects based on multiple variables

I am trying to determine the two most similar years based on a number of environmentally variables. For example, I would like to choose the most similar year to 2015, from the set 1989 to 2016, based ...
0
votes
0answers
10 views

Test for determining likelihood of finding disease in 1 of 2 lymph nodes in patients with cancer

I am performing a study looking at two specific lymph nodes in the head and I want to be able to say if disease is more likely to be present in 1 or the other lymph node. Also, if disease is present ...
0
votes
0answers
13 views

Uncertainty on the standard deviation of data set, when the data points are uncertain

I have a set of four data points (x,y), and I would like to describe the spread of the data about the mean in both the x-direction and in the y-direction. Each individual data point has an uncertainty ...
0
votes
0answers
10 views

Is using pairwise comparison instead of single label value a valid way to augment training data

There are many papers using siamese architecture to do pairwise ranking. For example: suppose my training set is $\{X,Y\}$, where $Y$ is the continuous value. I can find a regressor to minimize $\|h(X)...
0
votes
0answers
18 views

How to prepare data for anova or manova

I have a datafile (below is Gdrive link) and I want to prepare it for Anova/Manova tests. About Data: ...
0
votes
0answers
12 views

How to convert a Scikit-learn dataset to a Pandas dataset? [migrated]

How do I convert data from a Scikit-learn Bunch object to a Pandas DataFrame? ...
1
vote
0answers
14 views

Statistical Approaches for Uncertain Target Variable

Let me first explain what I have tried to mean by uncertain target variable. Consider the following scenario: Illegal immigrant behavior: Suppose you want to interdict with illegal immigrants to ...
2
votes
1answer
27 views

estimating variance using only data at the tails without resorting to Gibbs sampling

Suppose we know that the population size is $n=1,000$ but for whatever reason, we only have the bottom $n_1=100$ observations and the top $n_2 = 200$ observations. Furthermore, suppose we know the ...
0
votes
1answer
24 views

Data visualization for segments in a grid?

I'm working on a segmentation of individuals who have particular purchase behaviors. The net result is a set of 20 segments of people based on their consumption levels across two axes. An example ...
2
votes
0answers
19 views

Impact of inverting grayscale values on mnist dataset

http://yann.lecun.com/exdb/mnist/ Pixel values are 0 to 255. 0 means background (white), 255 means foreground (black). Is there a reason why the original ...
1
vote
1answer
92 views

Data Comparison in R [closed]

I have to compare two data sets to make sure that they are identical. They are massive data sets with millions of data points. Is this possible in R and if so how would I go about that? I am brand new ...
0
votes
2answers
79 views

Is log transforming square root transformed data a legitimate data transformation?

Is it legitimate to do a "double transformation" on data? Specifically, log transforming data which has already been square root transformed, or conversely, square root transforming log transformed ...
0
votes
0answers
51 views

What is the most effective way to get a baseline rating prediction from netflix rating data?

I am doing an assignment where I am working with some of the Netflix Challenge data. I have a database that maps movie ids to customer ratings of that movie and a database of customers mapped to ...
2
votes
0answers
67 views

build my own 'pickle' image dataset for CNN training

I am trying to use Python & Tensorflow to make Convolution Neural Network to classify images. I found some examples from the Udacity assignment4 for notMINST image set. Now my question is : I ...
0
votes
0answers
27 views

Using polychoric correlation with non normal ordinal data

I have ordinal data on scale 1-5 for detected pollutants in water (1 = detectable in small proportions; 5= detectable in higher proportions; also 0 was asaigned - not detectable). based on the ...
1
vote
0answers
19 views

Creating a blended dataset/transitioning from one predictive model to another

I'm working with two different models right now using a variable X as an input. One of the models is said to be applicable for values of X less than A, and one is said to be applicable for values of X ...
1
vote
0answers
24 views

Prediction with large inputs and outputs

Being a total newbie with machine learning, I thought I would seek your advice on a problem of mine. I'm looking for any leads, starting points ans help you could provide me. I'm looking to predict ...
1
vote
0answers
16 views

Labeling methylated/unmethylated sequences [closed]

I'm trying to do some test with matlab/SVM to classify methylated/unmethylated human sequences. I'm using these sequences freely avaible at this address: http://services.ibc.uni-stuttgart.de/name21/...
1
vote
0answers
21 views

How to convert standardized line graphs into a function?

I have graphed revenues from 4 different products of the same variety (ie. four different movies and the revenues they brought in). The axes are: x is the weeks and y is the revenues; with the lines ...