Structured data files in any format, collected together with the documentation that explains their production or use.
2
votes
0answers
31 views
Benchmark data for Random Forest evaluation
I would like to make/explore several alternative formulations of random forest (link) and am looking for the current state of the science for evaluating performance.
In the reference file Leo ...
0
votes
0answers
34 views
Order values of $x_1$ to achieve desired correlation with $x_2$. [duplicate]
Let $D$ be a data set containing $n$ instances of two variables $x_1$ and $x_2$. Further let $x_1$, $x_2$ be independent according to some measure like linear correlation or mutual information etc.
...
0
votes
0answers
10 views
How to load arrays with 3 dimension using R2WinBUGS? [migrated]
Since WinBUGS and R have different ways of organizing data for arrays, how should I organize the data when using R2WinBUGS so that the order is correct? Thanks!
2
votes
1answer
23 views
Comparing two data sets for similarity with variable data quality
I have to compare two data sets for similarity. I'm designing a piece of software which will help improve a drummers accuracy. The software displays a rhythm which the drummer then attempts to play.
...
0
votes
0answers
34 views
How to create a new vector from grouped data in R? [migrated]
Possible duplicate on SO: Computing a new variable using conditions of an existing data frame
I am trying to create a vector (e.g. z) from the group means. The vector will have a length equal to the ...
1
vote
0answers
19 views
Simulate LMM data with specific error values using R
I have a need to simulate data in R using the following:
Simulate data for 5 (or more) subjects, indexed by $i$, $(i=1,~...,~5)$ with $6$ observations per subject indexed by $t$ $(t=1,~2,~...,~6)$. ...
0
votes
0answers
16 views
Getting the degree of overlap between each pair of classes
In a dataset containing many classes, is there any way to get some values which indicate how much the data-points of two different classes $y_i$ and $y_j$ are overlapped ?
0
votes
1answer
30 views
Where to find datasets related to US elections at the individual level? [closed]
I am currently looking for datasets that I can use to predict how an individual voter will vote in a US election.
Ideally, the dataset that I am looking for includes socioeconomic features such as ...
2
votes
2answers
61 views
How can I convert my dataset into a new dataset which is adjusted for confounding covariates?
I have used software before to do linear regression and factor in/out the confounding variables, but what I would like to do is generate a new data set which is adjusted for the confounding variables. ...
0
votes
0answers
37 views
social network simulation
I need Geo-location based social network dataset . But I could not find any source where I can find the dataset. I need the post with location and the information of re sharing of that information ...
0
votes
2answers
42 views
How to detect a quasi separation problem for a data set?
Suppose that we have a two-column data set. One column consists of a hundred x=0 and a hundred x=1, whereas the other one consists of y's (1 or 0 response). Besides, suppose that the P(Y=1|X=0) = ...
1
vote
1answer
49 views
0
votes
1answer
37 views
Performance decay when testing on bootstrap data
I have a strategy that has a Sharpe ratio of 1.6 when back tested over the past 10 years. When I run this same strategy on re-sampled data, the performance of the strategy goes down to 1.32.
Should ...
1
vote
2answers
72 views
Split dataset randomly
I have a database with 500 records. I want to split these records to 75% and 25% *randomly*in order to use the different datasets for training and testing to machine learning algorithms. Does anyone ...
1
vote
0answers
30 views
What current methods for statistical disclosure limitation are best trade-offs between data privacy and data utility?
In many situations raw microdata is not released by institutions due to privacy preserving. Many techniques are used for protecting sensitive values in data. But many of them can destroy multivariate ...
3
votes
4answers
207 views
How to perform dimension reduction after doing PCA in R?
I have a big dataset of +- 40000 observations, each containing 784 variables. Because this dataset is extremely large I want to perform a dimension reduction.
Now everywhere I read that I can use ...
0
votes
0answers
21 views
Measure the relation/association between the outcome and the independent variables
I am running a model (logistic regression) with 20 independent variables in R.
Before running the model I calculated the correlation between all the variables and finally selected my variables by ...
-1
votes
0answers
21 views
Sports Stats API: Stats between dates [closed]
Looking to find an API to get sports stats (MLB, NFL) for athletes between certain dates. I.e. number of HR hit in the last 30 days. I've looked at the espn and yahoo APIs but they don't provide ...
0
votes
0answers
22 views
Comparing the magnitude of the standard deviation with the average value
For calculating the concentration of an unknown sample, my average value after three trials was about .03425 +- twice the standard deviation (.00159)
In comparing twice the standard deviation to the ...
-1
votes
1answer
59 views
Sports Statistics [closed]
I was wondering if anyone knew of any solid source of information on sports statistics (ie. Baseball statistics like rbi, hits, steals, etc. Basketball info like steals, three-pointers, wins, etc. ...
0
votes
0answers
49 views
Pearson Correlation Coefficient and gene expression matrix
This is a gene expression matrix. The first column are gene ids, the latter values are normalized gene expression values for controls (first 3 cols) and disease state (last 3 cols).
...
0
votes
0answers
27 views
Inference in a mix of two datasets with different features
The other day I was wondering this: Let's say that I have two experiments, a sample $N = N_1 \cup N_2$ and a set of features $F = F_1 \cup F_2$. In the experiment 1, from the $N_1$ sub sample are ...
1
vote
3answers
60 views
How to analyse three categorical variables
I need some help identifying a test to use for three categorical variables: Subject (maths, business etc), Big 5, and Learning style. I am carrying out research on whether there is a relationship ...
0
votes
0answers
21 views
R: Assigning variable to quintile on monthly basis [closed]
I am trying in R to indicate in which quintile a value of a variable is for every month of my data frame in this case based on volatility. For each month I want to know for each stock if it is in the ...
1
vote
0answers
31 views
Accuracy Measurement in Biased Dataset
Lets say we have data-set consists of 100 instance, each instance belong to one of 6 classes {1,2,3,4,5,6}. Furthermore, 70% of the instances belong to class 3.
For classification problem, if the ...
1
vote
0answers
42 views
Designing genomic search strategies for non-coding RNAs using BLAST
I am interested in developing a computational search of some types of non-coding RNAs (ncRNAs) on a new genome. I have some questions related to the nature of my experiment, because it is an ...
0
votes
0answers
26 views
0
votes
0answers
54 views
Data Mining: Association Analysis [closed]
Given dataset of transactions:
A B C D
B A
C D B
A B D
B D C
A C
where A, B, C and D are items.
a) How would I write all ...
0
votes
1answer
31 views
mapping data with a spike to a heat map
I have the following data set that I need to display on the heat map:
[ 30, 15, 66, 7, 9999, 78, 42, 132 ]
So if I map the values to the color scale using a ...
0
votes
0answers
38 views
Removing data above the mean
I am working with a data set, for which to remove noisy data I take the average of the sample itself and cut anything above that average. What I'm trying to understand here is does this have a name ...
2
votes
4answers
95 views
Strangely imbalanced dataset
I'm new to Machine Learning and this forum. I have a beginner's doubt regarding imbalanced dataset. Here it goes:
I have a binary classification task, where I'm more interested in accurately ...
1
vote
2answers
130 views
Finding patterns in data
I am probably looking for a definition.
Imagine we have 10 variables, but we are not interested in some kind of linear relation (nor quadratic or with any curve to it). What I would like is a way to ...
1
vote
0answers
10 views
Simulation of remission and after remission times under specific conditions
This is basically a data generation problem.
Say, $t$ is an exponential lifetime with mean two years. $tr$ is the remission time and $ts$ is the after remission time. So, $t=tr+ts$. I need to ...
0
votes
0answers
55 views
Get the percentage of two data sets compability
I'm new here and I don't have special background on math / stat. I'm a computer programmer and need to create a program that involved statistic.
I want to search the closest of two set data, which is ...
0
votes
0answers
43 views
Steps for Regression when subscales (based on survey responses)are present in IND and DEP variable
I am looking to run regression against 5 Independant variables (each comprised of 4-6 subscale survey items) and a Dependant Variable (comprised of 3 subscale survey questions in Likert Fashion). I ...
0
votes
0answers
17 views
What is record abstraction?
Is record abstraction basically the idea of coding variables? For example, coding location as (North = 1, South = 2, East =3, West = 4). Does record abstraction improve the quality of your data?
0
votes
0answers
101 views
Understanding Durbin-Watson in Panel Data
What is the real meaning of D-W stat. in a panel data? Does it matter if I don't report it in my paper?
1
vote
2answers
149 views
Is this outlier merely unusual or does it signify manipulation?
These are actual reported costs (60) for a series of projects for two years, say USD (though actually not) totalling $57,975,403.24:
...
0
votes
0answers
17 views
Data Structure for grouped queries?
I'm not sure what a more precise title could be. I have daily data incoming in which I want to total occurrences of a type. E.G today produced
...
1
vote
6answers
564 views
How to detect outliers in skewed data set?
I am working on my school datamining project. Within preprocessing stage I need to remove outliers from my data set which is positively skewed (see description). I have an idea to remove all values ...
0
votes
0answers
22 views
Attitude scaling
I wanted to know how to analyze the data to reflect the respondent's attitude. I 've got several questions or items about the trust in judicial system each of which has the responses 1. strongly agree ...
-1
votes
1answer
56 views
What are the statistic for the percentage of illegitimate baby in US? [closed]
Is there a way to know that? Is this a good stackexchange to ask?
Basically I am looking at the number of children within marriage whose biological dad is not the mother's husband during conception.
0
votes
0answers
45 views
Reliability of a rating scale
My aim is to evaluate the reliability and validity of a new observation rating scale (likert-type, with points 1 to 5) of items, representing different behaviours. The sample does not meet parametric ...
-1
votes
1answer
56 views
validating a regression model
I want to build a regression model to predict daily income from customers. I have 2 problems:
1. Choosing data for the training set - do I use daily income from 1 month ago, 6 month ago etc.
2. How do ...
0
votes
1answer
52 views
Constructing a measure versus using an interaction
My dataset is sports data and the outcome variable is wins in a season and I am trying to see the affect of the characteristics of the players on the team on this outcome. My question is, is it easier ...
2
votes
0answers
144 views
Example of classification dataset where SVM with linear kernel performs well
I'm looking for a dataset (preferably with a story, at any rate a real dataset)
where a SVM with a linear kernel performs well...in other words i'm looking for
a dataset where the class boundary is ...
0
votes
1answer
38 views
Analysis of a dataset and tool to get the incidence of an event
I have a dataset and I want to do statistical analysis:
...
-1
votes
1answer
29 views
Repository of datasets with rich metadata search / export [closed]
Are there some repository where one can query based on various metadata ?
For instance, if you go to uci datasets you can select based on the values of a few metadata.
Is there any other website ...
1
vote
0answers
48 views
CDF and confidence interval on non-parametric binomial data
I am having trouble with a series of system tests in which each test is assumed to be a normal approximations of binomial distribution. Each test sample, n, is a Bernoulli ("success", "failure") trial ...
0
votes
0answers
38 views
Question on re-indexing data for graphing
A file contains data for 1975-2012 of housing prices across a panel of countries.
However, it's indexed to 2005, so each country's housing price series converges to 100 in 2005, then they spread ...
