**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

1
vote
1answer
15 views

How to connect groups if they are not large enough?

I am new at statistics and I have problem that I have no idea how to solve. I have biological data, which includes expression levels (130 patients) for different alleles (I'm analysing SNP's) of ...
2
votes
2answers
11 views

Quantifying the irregularly of a data set

I have two 1D finite domains consisting of $N$ points each. In one domain the points are all regularly spaced, but in the other they are irregularly spaced. I would like to somehow quantify just how ...
0
votes
1answer
37 views

Machine Learning: What are the types of data set?

Is there a classification of the type of data sets in machine learning problems? I am specifically interested in classification problems. I know a large number of algorithms like SVM, NN, Decision ...
0
votes
0answers
12 views

Methods of statistics analysis to use with ecommerce data [closed]

I´m new in data analysis (i'm learning R) and want to do some analysis in a ecommerce data. I want to compare thing like: . Returning visitors X new visitors - Returning visitors spend more on the ...
0
votes
0answers
9 views

Ordering data - R and Rstudio [migrated]

I am still learning to use Statistical softwares and I have a question about R. I downloaded R and RStudio. I have a data table which has one column with 5571 codes, one code in each line. Each code ...
0
votes
0answers
4 views

Recognizing types of people on a CDR data set

I have 2 month of call detail record of people in my country. Each record has a user id, lon/lat location, and datetime. Some (About 1/4) users are classified as class A/B. by the mobility patterns I ...
3
votes
1answer
27 views

bar diagram or line diagram?

the population density of a country over 1950 to 2010 will have to represented by a suitable statistical diagram . which of the diagrams will be most suitable ? I have been given two options ...
0
votes
0answers
14 views

Best way to represent survival data

I have a question regarding the best way to represent survival data to predict the reliability of bike parts. I have the manufacturing date of each bike. At the manufacturing data the bike is ...
1
vote
1answer
52 views

How to approach a regression problem? [closed]

I have to solve a regression problem involving 302 variables. How do we select appropriate models or ensemble of models to work well ? Does this decision of choosing models come from the ideas ...
0
votes
0answers
3 views

Probability standard deviation random fluctuation in data

Let's say we have n vectors with k elements: $v^1$ = [$e^1_1 ,e^1_2,...e^1_k$] $v^2$ = [$e^2_1 ,e^2_2,...e^2_k$] and so on till $v^n$. Now let's suppose i have an algorithm that selects an element ...
1
vote
1answer
54 views

Will Linear Regression choose as good of a model as any other regression algorithm given enough data?

I am in the seemingly unusual situation of having practically unlimited data. In this case, will linear regression choose as good of a model as any other algorithm in the case where the number of ...
0
votes
0answers
15 views

Data Cleaning in R [migrated]

I have a csv file which I want to extract only the timestamp of the sentences which contain toward plus the fruit name in that sentence. How can I do this in R (or ...
0
votes
0answers
11 views

General knowledge for statistics (polling)

Assuming I did a survey and the following are the response received: Strong unsatisfactory(SU) : 200 Unsatisfactory (U) : 100 Neutral(N): 2 Satisfactory(S) : 100 Strong satisfactory(SS) : 200 ...
0
votes
1answer
19 views

Number of lags for a unit root test

I have some panel data and am suspicious of presence of unit root. As the dataset is unbalanced, I would like to use the xtunitroot fisher command in Stata. Is ...
1
vote
1answer
24 views

Histograms with similar means but different appearances

I have two histograms shown below. As you can see, the two histograms are different in appearance. However, when I calculate the mean of the two histograms, they are pretty similar. The mean for ...
0
votes
0answers
34 views

What statistical test should I use for binary data?

I am a slightly complex dataset, so I'll describe, I looked at medical visits from 2013 to 2014 and compared them to medical visits from 2013 - 2014. I documented each visit as a unique encounter. ...
0
votes
0answers
20 views

Which statistical test to use for comparing frequency of cells in four different sample groups?

I am currently writing up my thesis for my MSc and I wondered if someone could help with the type of statistical test I should be using and why. I am not very good at the statistics part and any help ...
0
votes
0answers
5 views

Differentiating given data sets

Is is possible to treat Data A and Data B the same in this case? Data A =One sample with before and after treatment responses e.g cattle live weight before castration and after castration. Data B =one ...
0
votes
0answers
6 views

help with using unstack in r [migrated]

I have a data frame that I wish to transpose from long to wide form. My advisor suggested using unstack and even wrote a bit of sample code to help me out. I'm able to run his sample code just fine ...
0
votes
0answers
19 views

How do I use a large amount of data to inform a failure probability analysis?

I'm trying to calculate a failure probability for a given $n$-dimensional model with a large data set. I've chosen FORM/SORM as tools, which require that I calculate a joint probability distribution ...
1
vote
0answers
35 views

95% confidence interval for an estimate average

I have a single dataset - from that dataset I have used four methods ($i=1, 2, 3, 4$) to estimate a parameter ($u_i$) and to generate a 95% confidence interval. In other words, I know $V[u_i]$ but not ...
1
vote
1answer
27 views

Email and IP String preprocessing for classification task

I am relatively new to the field of data-science, pardon my novice question. What are the available methods to convert email and ...
0
votes
0answers
7 views

Refrences for interval censored tobit (ICT) model

I am running an experiment called convex time budget where in I will be giving six options to the choose from. I want to exploit the interval nature of the data and want to use interval tobit to ...
1
vote
1answer
26 views

Which is the best technique to identify from which category a sample belongs

I'm a software developer and I'm facing a problem that I think should be solved through statistical techniques. We are developing software to classify some data. At the beginning of the project the ...
0
votes
0answers
5 views

R: reshape data with time-varying variables to long format [migrated]

I try to reshape my data from wide to long format. I have the following 3 example datasets. ...
0
votes
0answers
11 views

How should I test on a small dataset?

I use Weka to test machine learning algorithms on my dataset. I have 3800 rows and around 25 features. I am testing the combination of different features for prediction models and seem to predict ...
0
votes
0answers
10 views

Best-fit values for multiple data sets with single model

This is probably so simple the answer should be obvious, but I thought it best to check with better informed people than myself; I have a simple physics model that fits to some four data sets. From ...
0
votes
0answers
24 views

Data sets for which PCA can classify better than LDA (using a very small training set)

Can you provide an example of a dataset where PCA can find better discriminant directions than (LDA) Linear Discriminant Analysis? One example is UCI's wine data set. If you use only 2 observations ...
0
votes
0answers
10 views

ANN Training data set size for sparse vectors

I'm trying to approximate a function from $\mathbb{R}^n \to \mathbb{R}^m$ where $n$ is going to be on the order of $10^4$ or $10^5$ and $m$ is on the order of 10. The catch is that in the input the ...
0
votes
0answers
9 views

How does one call external datasets into scikit-learn? [migrated]

For example consider this dataset : (1) https://archive.ics.uci.edu/ml/machine-learning-databases/annealing/anneal.data or (2) http://data.worldbank.org/topic How does one call such external ...
0
votes
0answers
26 views

Where is the original data set for the “German tank problem”? [migrated]

I am looking for the original data set for: the "https://en.wikipedia.org/wiki/German_tank_problem" , but can't seem to find it. Any suggestions on where it might be available from? Thanks.
0
votes
0answers
32 views

Data Analysis in psychiatric

A 30 item questionnaire in accept or decline form consisting of commonly encountered dysfunctional beliefs about psychiatric issues was administered to medical students. The questionnaire is in the ...
0
votes
1answer
25 views

compare multiple algorithms in the same dataset

I want to do a comparison between multiple algorithms when they applied to the same dataset. For example, I have a set of texts in Arabic and another version translated to English. I did a sentiment ...
1
vote
0answers
17 views

advice about the suitable ARIMA model

I am trying to do seasonal ARIMA(0,1,1) model using matlab But I am a little confused about the results. The drawing of standard normal quantils and the standardized residuals are shown below: my ...
2
votes
2answers
51 views

Average of a financial ratio: which average

A financial ratio, like e.g., return on equity (ROE) is a division of two numbers ROE=return/equity. If I analyse a whole sector then one can compute the ROE for each company in that sector and thus ...
0
votes
0answers
8 views

What are word components in IAM db3 for xml

I'm using the IAM handwritten database and for the answer keys I get the hierarchy, however I'm confused about word components in the the handwritten section. Specifically seeing something like this ...
0
votes
1answer
69 views

Creating a test set with imbalanced data

I am working on a binary random forest using R. mu data set consists of 300 cases classes 1 and 2100 cases class 0. I am planning to evaluate my model using the model prediction and the AUC and for ...
0
votes
0answers
12 views

Trimming, Winsorising and manipulating biological data

So I have an experiment with multiple groups (groups 1-6), individuals in each group (n=10), and read outs per individual. Eg. below Group 1: 28.1, 28.6, 29, 30.3,30.4,30.6,30.7,31.1.34 Group ...
0
votes
0answers
26 views

Problem with data on Matlab Neural Network Toolbox

I am currently working on a NARX network for a time-series prediction problem. I am using a 3*4644 array as inputs and a 1*4644 array as my targets. I do not have any delay on the input and the ...
0
votes
0answers
21 views

Type one right censored data [duplicate]

Does anyone have any idea about how to simulate censored data in R? For example, generate a sample of 200 with 30% of that Type 1 right censored, based on Weibull distribution. (30% of 200 = 60).
2
votes
2answers
34 views

Continue the predictions beyond the current data using time series neural network

I have a single time series variable and I want to train a neural network in a sort of auto-regressive fashion. specifically, I have data for water consumption that is changing with time In the ...
0
votes
0answers
45 views

Comparison of 2 data sets with different sample sizes

I have two data sets that have different sample sizes and I am unsure on how to compare them. Dataset 1 has 51 participants but dataset 2 has only 20 participants. The answers to the questions are ...
2
votes
0answers
31 views

When is a single data point enough to be considered a trend in the data?

In the general case, when collecting data points at disparate points in time (such as once per day or two), under what conditions can a single low data point be considered a downwards trend in the ...
0
votes
0answers
12 views

how do I split my data for EM?

I'm going to use EM to replace my missing values, as my missing data is MCAR and less than 1% of the total. My variables are from several measures, taken at two time points, and from both a parent and ...
0
votes
1answer
25 views

Dependent questions in SPSS [closed]

I have questions in a survey that are dependent on other questions, i.e. nested questions. How do I enter those into SPSS? Example: ...
0
votes
0answers
13 views

Determining fit for potentially periodic non-timeseries data?

I am working with genetic data that has 100,000 wide regions with given diversity values attached to them, as below: ...
1
vote
1answer
45 views

PCA dataset to test my implementation

I'm trying to find a dataset with examples of data matrices and their principal components. I know that I could construct my own dataset using standard implementations found in python, or octave ...
0
votes
0answers
49 views

Obfuscating sensitive data keeping data-properties intact

I am preparing a dataset for my academic interests. The original dataset contains sensitive information from transactions, like Credit card no, ...
1
vote
1answer
14 views

sklearn estimate_bandwidth in MeanShift algorithm with small dataset

How should I approach the estimate_bandwidth function with only very small datasets? My problem is that I am analyzing datasets that range from 500 entries to 2. ...
3
votes
1answer
64 views

Does Hashing data affect linear separability

What would be a good way to obfuscate sensitive information and store e-commerce transaction data, to later perform fraud analysis on it. One idea that crossed my mind, was to hash each sensitive ...