Structured data files in any format, collected together with the documentation that explains their production or use.

learn more… | top users | synonyms (1)

0
votes
1answer
19 views

IBM Synthetic Data generator

I am implementing some algorithms in association rule hiding. I need to apply these algorithms in different data sets. I searched for IBM synthetic data generator through Google but I could not find ...
0
votes
0answers
7 views

longitudinal data, functional data, high frequency data

I know what is longitudinal data and functional data. But I really do not know what's the exact definition for high frequency data (for example in finance). If high frequency just means the denseness ...
1
vote
1answer
20 views

Best way to visualise impact distance against launch speeds and angle

I am modelling the trajectory of a projectile, incorporating atmospheric drag. I am calculating the distance from the launch point for when the projectile impacts the ground. I have generated this ...
0
votes
0answers
4 views

Amplification using data sets [closed]

Are there techniques whereby I can apply a large data set to a very unrelated small data set and see if there are any patterns that can be identified?
-1
votes
0answers
29 views

Data Extraction - Historical Weather dataset [closed]

I want to get worldwide weather dataset from the year 2008 to 2014!... Where do I'll find this data? Can anyone suggest me the best website to extract the data or any other ways to obtain this ...
1
vote
1answer
28 views

K means and hierarchical

I have some insurance data - 5,00,000 records with around 50 variables :categorical,continuous and binary variables.Some of my categorical variables have 30+ categories.I am trying to build clusters ...
0
votes
1answer
25 views

Where do I get open data on open source projects? [closed]

I'm trying to find out how many open source projects related to hydrological/environmental modelling are out there, and derive some basic information (e.g. total number, deployments per year, etc.). ...
0
votes
0answers
18 views

applying statistics in “filling in” a data set?

Let me first say that my professional role is that of a mid level OLAP/OLTP developer (lesser experence with OLTP). Ive only had business statistics in school many years ago and I am currently taking ...
1
vote
0answers
12 views

Filtering noise in a data set

I'm trying to filter out noise from a Sonar. The idea is that the sonar in aimed upwards, after an object has come in it's range, I want to be able to tell if the object is moving away or coming ...
0
votes
0answers
10 views

Intrinsic topology and metrics… (looking for name of a method)

Suppose I have an n-dimensional dataset and its points are roughly in the shape of an n-dimensional horseshoe or something along those lines. Using euclidian distance might be a bad idea, since points ...
0
votes
0answers
42 views

R subsetting and factor levels

I was wondering why it is that when subsetting a dataframe, the new frame retains the factor levels of the parent dataframe. For example, consider an arcaheological data set that includes ...
2
votes
0answers
30 views

Data set structure

I'm having problem to understand how my dataset should be organized. I have unbalanced panel data with observations of products' prices as independent variable. Data is from a 3-year period. ...
0
votes
0answers
22 views

How much can I restrict my data with outliers?

I know there are tons of questions on CV.SE about outliers, but I didn't find a solution to my specific case. I have a dataset that I'm analyzing where in order to achieve "good" results, the data ...
0
votes
0answers
6 views

How to specify “unlimited” value

How to specify "unlimited" value of a variable in my dataset for regression purposes? I have a variable which can have both finite or unlimited amount of units.
0
votes
0answers
14 views

Can linear regression model variables be constantly looped in simulations to find the perfect model? [duplicate]

Say that I have a very large dataset and have a fairly large amount of variables, let's say 30. Since I don't know which variables matter and are good predictors for regression, I construct a for-loop ...
0
votes
0answers
32 views

Highlight a single data point on a graph with ~1800 (R) [migrated]

I know that the row of data I want is row 1535. plot(data.exoplanets$loga, data.exoplanets$logMass, ylab="Log of Mass", xlab="Log of Semi Major Axis") is the ...
0
votes
0answers
12 views

Partial least squares structural equation modeling: simulate dataset based on a given model

Is there any R package or other software, with which one can generate a dataset based on a given pls structural equation model? For covariance based structural equation modeling I found the simsem ...
-1
votes
2answers
124 views

can a software engineer be good at data analytics using statistics and R [closed]

I am a software engineer and new to R. I started learning but at many steps I end up learning stats concepts. So I was wondering if it's a good idea for a software developer to move to data ...
1
vote
2answers
47 views

How to normalize data without an edge

Is there some calculation method which normalizes data (numbers) if I don't have minimum and maximum edge? Only one I know is: y = (x - min-x) / (max-x - min-x). But as I said I don't have an edge ...
3
votes
2answers
96 views

How can I measure fluctuation?

For a certain product I have a data series with its price for each day over a long period of time (20 years). I want to investigate how the fluctuation of the price changes over the whole time. ...
2
votes
2answers
68 views

Calculate data distribution formula from data points

I have 50.000 pieces of data in a list. 10 random samples: 4,877; 4,165; 6,152; 5,544; 6,942; 7,409; 6,415; 5,592; 6,947; 4,965; 4,688 I'm trying to do 2 ...
0
votes
0answers
24 views

Resource for practice data analytics projects

does anyone know of resources that offer data analytics/mining projects that I can tackle to help build my portfolio and present to interviewers? Thank you.
7
votes
0answers
74 views

Are data handling errors already 'priced in' to statistical analysis?

Ok, fair warning--this is a philosophical question that involves no numbers. I've been thinking a lot about how errors creep into data sets over time and how that should be treated by analysts--or if ...
0
votes
1answer
43 views

Data analysis in a comparative way

I'm trying to make data analysis to huge amount of data of psychophysical task. Well, participant should estimate if a heared sound belong to which frequency (we have 8 sounds). So for a played sound ...
3
votes
1answer
27 views

Is it better to pre-filter the entire data set or just the training sub-set?

I am currently working on a classifier for the qualitative spectral analysis of alloys. One of the problems that I faced is preparation of samples for the classifier training. Samples have to me ...
-3
votes
0answers
21 views

Help to interpret data set

I am having trouble interpreting the data set in the following link: http://ise.tamu.edu/people/faculty/butenko/market/ does each datamatrix*.txt file contains prices of financial instruments ...
1
vote
0answers
31 views

Correlation and coefficient with overlapping data

I have a time series data, with 5-period rates sampled at 1 period intervals. Essentially $$r_i = x_{i+5} - x_i ; ~~ i=1,...,N-5$$ This creates an overlapping data problem for regression. As far ...
0
votes
1answer
100 views

Generating text data for training for doing named entity recognition and extraction

I'm trying to build an algorithm for doing named entity extraction. It goes like this. There is a large set of text documents [communications], from which specific information has to be extracted. The ...
0
votes
1answer
43 views

How to standardize text data for training Neural Networks?

I want to train neural network with text data(natural language) as input for classification purpose. One way for standardizing text data for neural network is to use N-GRAM/SKIP-GRAM representation ...
0
votes
0answers
24 views

Analysing multiple response question

Suppose I have a survey data set, where sample size is n=50. There is a variable on " What do you like about this product? " Answers: a. Less time consuming b. Low weight c. User ...
1
vote
0answers
39 views

Multiple imputation in SPSS: merge datasets?

I have two sets of data, measured from the same persons. The variables in the two sets are very different. There are some extreme values (occurring at random) in both sets I would like to handle using ...
4
votes
2answers
25 views

Do any statistical tests compare distributions when one has significantly fewer samples?

I've been researching different methods to compare two distributions for equality, or inequality rather. I want to compare actual user performance against projected performance, after a particular ...
2
votes
0answers
157 views

Linear mixed models or not?

We did a study with a course participants where we have one outcome variable and several metrics calculated for each message they posted on a discussion forum. Basically, the data are structure in the ...
1
vote
1answer
19 views

Is there any repository with interval censored time-to-event datasets?

I'm looking for this particular structure of data for working on my thesis. In particular, I need interval censored with a cure fraction data. This kind is actually popular in medicine and clinical ...
0
votes
0answers
11 views

Good mean shift clustering datasets

I am working on a modification of the Mean Shift algorithm and would like to validate my clustering results. I am struggling to find suitable clustering datasets to compare performance with. I'm ...
0
votes
0answers
16 views

What can I possibly get from my data to find what I need?

I am an end-user of a program. This program returns an output based on an input (a file + a database). I have made a few tests with differents files and databases, then constructed a CSV file of the ...
3
votes
3answers
354 views

How to think about correlation

I have data that looks like this (I am using R): ...
1
vote
0answers
13 views

Variable selection multiple data types

I'm working on a data set containing 45 predictors that cover metric, ordinal and nominal variables. Aiming to finally build a model, which explains the interacting effect of a set of predictor ...
0
votes
2answers
68 views

If the model fits well, nothing can be done?

I am playing a data without any background information. First, I try multiple linear regression. The model fits well, since the $r^2$ is larger than 90%. I deleted several variables by AIC. The fits ...
2
votes
0answers
34 views

Finding the median distribution of many distributions?

I have many customers. Each generates his own distribution of shopping times. Each customer thus has their own empirical distribution of shopping times. If I want to get a sense of the average ...
1
vote
0answers
47 views

Data normalization for a recommender

this question is a copy from DataScience forum. I do hope I will get much more information and help here. At the moment I am doing some data experiments with the Graphlab toolkit. First of all, I ...
2
votes
1answer
53 views

How to interpret a status other than alive or dead in data for survival analysis?

I am working on a famous data set from this book. The data set consists of measurements on 418 patients. I am interested on modelling the variable; futime: number of days between registration ...
1
vote
0answers
26 views

What is robust data?

This is the rather confusing go-to internet definition for robust data: Robust data is data that is constructed to survive and function in multiple settings. It's reusable. It can be updated. Anyone ...
1
vote
0answers
12 views

Find out whether the answer to a question depends on a group

I've distributed a questionnaire to four different groups of people. An example of a question in the questionnaire is: Do you want to be informed about financial topics through social media? Yes/no. ...
1
vote
1answer
35 views

Potatoes in space! - Correlating sets of unordered 4d data

(I asked this question some time back but got no answers or comments. It's a pretty big question so therefore I thought dividing it into several questions might make it easier to answer. The original ...
1
vote
0answers
10 views

Adding exogenous input to data (to expand ARMAX)

I am facing the following problems: 1) I am trying to add an exogenous input to an existing ARMAX model. I already identified a good fitting model with small MSE for my system. ...
1
vote
1answer
29 views

Splines and clinical dataset query

Perhaps off topic or not directly suitable for crossvalidated but does anyone know of a good dataset with a clinical aspect I could use to teach myself how to use regression splines (in R)? I always ...
0
votes
1answer
36 views

Fitting a mixture model to spatially correlated data

When the data are spatially correlated, is the usual GMM likelihood function overweighted? The data. Scattering experiment, sensor is like a CCD. Can't see individual events, only density estimate ...
0
votes
1answer
30 views

Similarity datasets

Given two (or maybe more) datasets with the same samples/members, but with different variables. Is there a general way to compare the information available in the two datasets without looking into the ...
0
votes
0answers
22 views

Missing Values in Dependent Variable and the Question on Tobit model (Panel Data)

I would appreciate some help and guidance on this issue. I have a panel data that looks like this. What I want to do is use the FDIflow variable as a dependent ...