**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

2
votes
1answer
23 views

Do you merge your validation and training sets?

I'm currently implementing a K-Nearest Neighbours model and I'm at the stage of splitting up the datasets for cross-validation. I understand the need for the Training, Validation and Test sets, ...
0
votes
0answers
4 views

Likert scale in SPSS analysis

my study hypothesis is that patients perceive their antiretrovirals treatment differently as opposed to their cotreatments and have better adherence to their antiretrovirals than to their ...
0
votes
0answers
9 views

How do I remove the header names in a vector? [on hold]

I have a character vector which looks like this: ` NA1 NA2 NA1 NA2 NA1 NA2 "4e" "41" "4e" "41" "4e" "41" ` I want to remove the NA1,NA2,.... headers ...
0
votes
0answers
18 views

How do I encrypt specific variables/columns in a data frame in R? [migrated]

There are certain variables in my data frame which need to be encrypted because it contains sensitive data.How do I go about encrypting these columns/variables?
0
votes
0answers
14 views

Using KNN or Clustering techniques to increase data sample size

had a question regarding using KNN or clustering techniques to 'pad' smaller data sets with similar data points. Say, I have some data, a modified and simplified snippet of which looks like this: ...
0
votes
0answers
10 views

Fitting data with interdependence

I have an issue, of which I'm not sure how can be solved. I'm trying to do so with Mathematica (and their stack exchange recommended I try here), but other solutions are also possible of course. ...
8
votes
3answers
463 views

What are some good datasets to learn basic machine learning algorithms and why?

I am new to machine learning and looking for some datasets through which i can compare and contrasts the differences between different machine learning algorithms (Decision Trees, Boosting, SVM and ...
0
votes
0answers
10 views

Data complexity measure which affect classifier performance

As we strive to explain accuracy of machine learning algorithms, many authors suggest to start by degree of complexity in data. I am working in data complexity measure like: class ...
1
vote
1answer
16 views

How to connect groups if they are not large enough?

I am new at statistics and I have problem that I have no idea how to solve. I have biological data, which includes expression levels (130 patients) for different alleles (I'm analysing SNP's) of ...
2
votes
2answers
12 views

Quantifying the irregularly of a data set

I have two 1D finite domains consisting of $N$ points each. In one domain the points are all regularly spaced, but in the other they are irregularly spaced. I would like to somehow quantify just how ...
0
votes
1answer
38 views

Machine Learning: What are the types of data set?

Is there a classification of the type of data sets in machine learning problems? I am specifically interested in classification problems. I know a large number of algorithms like SVM, NN, Decision ...
0
votes
0answers
9 views

Ordering data - R and Rstudio [migrated]

I am still learning to use Statistical softwares and I have a question about R. I downloaded R and RStudio. I have a data table which has one column with 5571 codes, one code in each line. Each code ...
0
votes
0answers
5 views

Recognizing types of people on a CDR data set

I have 2 month of call detail record of people in my country. Each record has a user id, lon/lat location, and datetime. Some (About 1/4) users are classified as class A/B. by the mobility patterns I ...
3
votes
1answer
29 views

bar diagram or line diagram?

the population density of a country over 1950 to 2010 will have to represented by a suitable statistical diagram . which of the diagrams will be most suitable ? I have been given two options ...
0
votes
0answers
17 views

Best way to represent survival data

I have a question regarding the best way to represent survival data to predict the reliability of bike parts. I have the manufacturing date of each bike. At the manufacturing data the bike is ...
1
vote
1answer
52 views

How to approach a regression problem? [closed]

I have to solve a regression problem involving 302 variables. How do we select appropriate models or ensemble of models to work well ? Does this decision of choosing models come from the ideas ...
0
votes
0answers
4 views

Probability standard deviation random fluctuation in data

Let's say we have n vectors with k elements: $v^1$ = [$e^1_1 ,e^1_2,...e^1_k$] $v^2$ = [$e^2_1 ,e^2_2,...e^2_k$] and so on till $v^n$. Now let's suppose i have an algorithm that selects an element ...
1
vote
1answer
54 views

Will Linear Regression choose as good of a model as any other regression algorithm given enough data?

I am in the seemingly unusual situation of having practically unlimited data. In this case, will linear regression choose as good of a model as any other algorithm in the case where the number of ...
0
votes
0answers
15 views

Data Cleaning in R [migrated]

I have a csv file which I want to extract only the timestamp of the sentences which contain toward plus the fruit name in that sentence. How can I do this in R (or ...
0
votes
0answers
11 views

General knowledge for statistics (polling)

Assuming I did a survey and the following are the response received: Strong unsatisfactory(SU) : 200 Unsatisfactory (U) : 100 Neutral(N): 2 Satisfactory(S) : 100 Strong satisfactory(SS) : 200 ...
0
votes
1answer
19 views

Number of lags for a unit root test

I have some panel data and am suspicious of presence of unit root. As the dataset is unbalanced, I would like to use the xtunitroot fisher command in Stata. Is ...
1
vote
1answer
25 views

Histograms with similar means but different appearances

I have two histograms shown below. As you can see, the two histograms are different in appearance. However, when I calculate the mean of the two histograms, they are pretty similar. The mean for ...
0
votes
0answers
35 views

What statistical test should I use for binary data?

I am a slightly complex dataset, so I'll describe, I looked at medical visits from 2013 to 2014 and compared them to medical visits from 2013 - 2014. I documented each visit as a unique encounter. ...
0
votes
0answers
20 views

Which statistical test to use for comparing frequency of cells in four different sample groups?

I am currently writing up my thesis for my MSc and I wondered if someone could help with the type of statistical test I should be using and why. I am not very good at the statistics part and any help ...
0
votes
0answers
5 views

Differentiating given data sets

Is is possible to treat Data A and Data B the same in this case? Data A =One sample with before and after treatment responses e.g cattle live weight before castration and after castration. Data B =one ...
0
votes
0answers
6 views

help with using unstack in r [migrated]

I have a data frame that I wish to transpose from long to wide form. My advisor suggested using unstack and even wrote a bit of sample code to help me out. I'm able to run his sample code just fine ...
0
votes
0answers
20 views

How do I use a large amount of data to inform a failure probability analysis?

I'm trying to calculate a failure probability for a given $n$-dimensional model with a large data set. I've chosen FORM/SORM as tools, which require that I calculate a joint probability distribution ...
1
vote
0answers
35 views

95% confidence interval for an estimate average

I have a single dataset - from that dataset I have used four methods ($i=1, 2, 3, 4$) to estimate a parameter ($u_i$) and to generate a 95% confidence interval. In other words, I know $V[u_i]$ but not ...
1
vote
1answer
27 views

Email and IP String preprocessing for classification task

I am relatively new to the field of data-science, pardon my novice question. What are the available methods to convert email and ...
0
votes
0answers
7 views

Refrences for interval censored tobit (ICT) model

I am running an experiment called convex time budget where in I will be giving six options to the choose from. I want to exploit the interval nature of the data and want to use interval tobit to ...
1
vote
1answer
26 views

Which is the best technique to identify from which category a sample belongs

I'm a software developer and I'm facing a problem that I think should be solved through statistical techniques. We are developing software to classify some data. At the beginning of the project the ...
0
votes
0answers
5 views

R: reshape data with time-varying variables to long format [migrated]

I try to reshape my data from wide to long format. I have the following 3 example datasets. ...
0
votes
0answers
12 views

How should I test on a small dataset?

I use Weka to test machine learning algorithms on my dataset. I have 3800 rows and around 25 features. I am testing the combination of different features for prediction models and seem to predict ...
0
votes
0answers
10 views

Best-fit values for multiple data sets with single model

This is probably so simple the answer should be obvious, but I thought it best to check with better informed people than myself; I have a simple physics model that fits to some four data sets. From ...
0
votes
0answers
24 views

Data sets for which PCA can classify better than LDA (using a very small training set)

Can you provide an example of a dataset where PCA can find better discriminant directions than (LDA) Linear Discriminant Analysis? One example is UCI's wine data set. If you use only 2 observations ...
0
votes
0answers
10 views

ANN Training data set size for sparse vectors

I'm trying to approximate a function from $\mathbb{R}^n \to \mathbb{R}^m$ where $n$ is going to be on the order of $10^4$ or $10^5$ and $m$ is on the order of 10. The catch is that in the input the ...
0
votes
0answers
9 views

How does one call external datasets into scikit-learn? [migrated]

For example consider this dataset : (1) https://archive.ics.uci.edu/ml/machine-learning-databases/annealing/anneal.data or (2) http://data.worldbank.org/topic How does one call such external ...
0
votes
0answers
26 views

Where is the original data set for the “German tank problem”? [migrated]

I am looking for the original data set for: the "https://en.wikipedia.org/wiki/German_tank_problem" , but can't seem to find it. Any suggestions on where it might be available from? Thanks.
0
votes
0answers
32 views

Data Analysis in psychiatric

A 30 item questionnaire in accept or decline form consisting of commonly encountered dysfunctional beliefs about psychiatric issues was administered to medical students. The questionnaire is in the ...
0
votes
1answer
26 views

compare multiple algorithms in the same dataset

I want to do a comparison between multiple algorithms when they applied to the same dataset. For example, I have a set of texts in Arabic and another version translated to English. I did a sentiment ...
1
vote
0answers
17 views

advice about the suitable ARIMA model

I am trying to do seasonal ARIMA(0,1,1) model using matlab But I am a little confused about the results. The drawing of standard normal quantils and the standardized residuals are shown below: my ...
2
votes
2answers
52 views

Average of a financial ratio: which average

A financial ratio, like e.g., return on equity (ROE) is a division of two numbers ROE=return/equity. If I analyse a whole sector then one can compute the ROE for each company in that sector and thus ...
0
votes
0answers
8 views

What are word components in IAM db3 for xml

I'm using the IAM handwritten database and for the answer keys I get the hierarchy, however I'm confused about word components in the the handwritten section. Specifically seeing something like this ...
0
votes
1answer
69 views

Creating a test set with imbalanced data

I am working on a binary random forest using R. mu data set consists of 300 cases classes 1 and 2100 cases class 0. I am planning to evaluate my model using the model prediction and the AUC and for ...
0
votes
0answers
12 views

Trimming, Winsorising and manipulating biological data

So I have an experiment with multiple groups (groups 1-6), individuals in each group (n=10), and read outs per individual. Eg. below Group 1: 28.1, 28.6, 29, 30.3,30.4,30.6,30.7,31.1.34 Group ...
0
votes
0answers
30 views

Problem with data on Matlab Neural Network Toolbox

I am currently working on a NARX network for a time-series prediction problem. I am using a 3*4644 array as inputs and a 1*4644 array as my targets. I do not have any delay on the input and the ...
0
votes
0answers
21 views

Type one right censored data [duplicate]

Does anyone have any idea about how to simulate censored data in R? For example, generate a sample of 200 with 30% of that Type 1 right censored, based on Weibull distribution. (30% of 200 = 60).
2
votes
2answers
36 views

Continue the predictions beyond the current data using time series neural network

I have a single time series variable and I want to train a neural network in a sort of auto-regressive fashion. specifically, I have data for water consumption that is changing with time In the ...
0
votes
0answers
52 views

Comparison of 2 data sets with different sample sizes

I have two data sets that have different sample sizes and I am unsure on how to compare them. Dataset 1 has 51 participants but dataset 2 has only 20 participants. The answers to the questions are ...
2
votes
0answers
31 views

When is a single data point enough to be considered a trend in the data?

In the general case, when collecting data points at disparate points in time (such as once per day or two), under what conditions can a single low data point be considered a downwards trend in the ...