**Requests for datasets are off-topic on this site.** Use this tag for questions concerning creating, processing, or maintaining datasets.

learn more… | top users | synonyms (1)

0
votes
0answers
22 views

Questionnaire data analysis. Correlation between a model and questionnaire data

Kind of a newbie with data analysis here, and looking for some guidance. I've made a questionnaire in relation to a model regarding the security of certain software. The questionnaire is mostly made ...
-3
votes
0answers
26 views

R. Change som data points (In one column I want to change all the 0's to1) [on hold]

I have a dataset in R. In one column I want to change all the 0's to 1. How do I do that. Some kind of 'if' loop I suppose?
1
vote
0answers
20 views

Google Trends: Stitching 90 day periods of daily data together

Google Trends lets you see the amount of researches made on for a term on google during a set period of time, normalized between 0 and 100 (depending on the highest value in that period). I would like ...
0
votes
0answers
11 views

Looking for an algorithm to generate (dummy) share price data [migrated]

Is there an easy-ish way I can generate "dummy" share price data for the purposes of data visualisation techniques etc.? Essentially I want to have something like the "Adventure Works" of price data. ...
0
votes
0answers
6 views

using panel data for for evaluation of policy [closed]

I have a unbalanced panel data for years 2004 and 2011. A policy introduced in 2007, I would like to find out policy impacts. How do i model treatment variable in panel data analysis?
0
votes
0answers
22 views

Clustering with restrictions

I have this dataset of a ranking of roughly 32.5k players in increasing order. The third column contains the number of players which have the corresponding score, whilst the second is the number ...
1
vote
0answers
17 views

Similarity between sets with different size

is there a distance measure like jaccard for sets with different sizes? For example A=['a','b','c'] and B=['a','d'] I would like to include the total intersection as well as the order. The ...
3
votes
1answer
34 views

How to analyze this positively skewed data?

I am having trouble analyzing my dataset consisting of the sumscores of a questionnaire. For each item, subjects had to indicate whether they performed this behavior 'never', 'sometimes', or 'often', ...
0
votes
1answer
11 views

Estimate the two groups in a panel data estimations simultaneously

how can I estimate two groups of countries in a data estimation panel simultaneously? (In order to compare the results between two groups, in one estimate). I should note that I am not allowed to give ...
-1
votes
1answer
23 views

Big data set for Document Classification [closed]

I'm looking for big data set which is suitable to be used for document classification task. The data set which I'm looking for should composed of the frequency of the words which exist in each ...
1
vote
0answers
23 views

The statistical coffee test

I have had a couple of courses in mathematics, and two in basic statistics. over the last few days I have pondered on how to perform the best possible test of coffee brands, using my knowledge of ...
1
vote
1answer
72 views

how to handle small datasets with large dimensions

I have 48 samples which are case and control and 27000 features for each sample so my matrix is [48 X 27000]and I am using Deep belief networks(DBN) as my algorithm to predict the accuracy of the ...
0
votes
0answers
27 views

Normalise heart rate data

I need to statistically examine time series of heart rate data over many different users. Since I don't have data for each user for each instance, I should find a way to normalise the data so that I ...
0
votes
0answers
23 views

How much data is needed for GLM?

I'm doing 2 different GLMs. A Poisson GLM for claim counts and a Gamma GLM for claim amounts. I've read a lot of literature about GLMs but I still haven't found a satisfying answer to how much data is ...
0
votes
0answers
5 views

Implementation for Association Rule except market basket analysis and medical

I wanna make application and the toool analysis is Apriori and apriorihybrid. But, this application is used to be in data census and survey. How to implement it in that data. because we usually use ...
1
vote
0answers
12 views

Detect grid in 2D plane

Given is a set of points (x,y) in a two dimensional Cartesian coordinate system. Many points are near the crossing points of an affine transformed grid. The points have been measured and may have: ...
0
votes
0answers
8 views

Visualization for discrete integers on a non-fixed domain

So I am wondering if someone can give some input on how I might go creating some good visualization for this data set I have. Lets say that I am allocating buffers in some code that I have and that ...
0
votes
1answer
17 views

Using a predictive model on a dataset with different explanatory variables

As part of an assignment, we built a logistic regression model on a training data set. The test data, however, doesn't have one of the variables that was in the training data and included in the ...
0
votes
1answer
49 views

Comparing two datasets

I'm not very familiar in statistics so please bear with me. I have two datasets which consist of four attributes: ...
1
vote
1answer
17 views

Can you train a classifier to identify anomalous conditions by only training on normal conditions?

I was having a discussion with a coworker with regard to machine learning classifiers--specifically we were thinking about Random Forest but that is somewhat besides the point. The use case we were ...
4
votes
1answer
51 views

Linear algebra attributes of a typical data set

Background / motivation: I was re-listening to Prof. Strang's video lectures on linear algebra addressing the $Ax = b$ matrix-vector multiplication, and I started to realize that, although intuitive, ...
0
votes
0answers
3 views

Opened itemset in data mining

Can you give me an example for an opened itemset in a data set D? A closed itemset is defined as follows: "An itemset X is closed in a data set D if there exists no proper super-itemset Y such that ...
2
votes
0answers
24 views

How to get the N th row from data frame in R [migrated]

For example, I have the following data frame: ...
0
votes
2answers
46 views

Most probable value given a dataset

Forgive my high-school level mathematics but say I have the following dataset: [2, 3, 4, 5, 6, 6.5, 7, 7.5, 8.5, 9.5] I want to know what value is most likely to ...
0
votes
2answers
29 views

Using missing data

I conducted a web-based survey. Started with 1000+ respondents and it gradually reduced to 400+. Some key questions were answered, for example key question re dependent variable is answered by N=750. ...
5
votes
1answer
44 views

Is it better to do exploratory data analysis on the train dataset only?

I'm doing exploratory data analysis (EDA) on a dataset. Then I will select some features to predict a dependent variable. The question is: Should I do the EDA on my training dataset only? Or ...
0
votes
1answer
21 views

Count Models for Continuous variables

I want to analyze the effect of taxes on the amount of fixed assets an affiliate of an multinational capitalizes. Fixed assets are of course not a count variable. Can I anyway use a count model such ...
1
vote
0answers
23 views

Why do classification tress tend to perform better on large data sets and logistic regression on smaller ones?

Is this simply because if we have a lot of data, there is a bigger chance that a more complicated decision boundary (boundaries) is required to separate the observations? Does someone know researches ...
2
votes
1answer
135 views

Is it necessary to split dataset for cross validation?

I am using caret package in R for training dataset and cross validation process. I am confused about cross validation process. Now, i am splitting the dataset to two subset, training and testing; ...
1
vote
0answers
22 views

Structuring Data for Logistic Regression - Probability of Attrition Model

I am struggling with the best method for setting up my data for building a probability of attrition model and was hoping I could get some help/ideas here. Basically, I have a very large dataset ...
0
votes
0answers
48 views

Mixture of beta distrubutions

I have data; one column and 310 rows. When I plot the histogram, it has two peaks; see the figure. I would like to find appropriate distribution that fits the data. I tried to mixtools in R, but it ...
0
votes
0answers
16 views

Suitable Estimator for Panel ARDL

Any body knows what are suitable estimators available to estimate panel ARDL(p,q) with N = 6 & T = 48? I tried to use mean group and pooled mean group estimator by Pesaran and Shin (1999) by ...
0
votes
1answer
48 views

Empirical formula- 68%, 95.5%, & 99.9% of scores

I'm having trouble with some stats questions for my psychological testing paper. Have tried emailing the professor but to no avail. Given the standard deviation of 19.8,Identify the range of scores ...
1
vote
0answers
9 views

What metric should I be gathering to allow making predictions about how dependent two variables are?

I currently have a large set of tuples (a,b) and have various statistics gathered independently about the set of a's and b's. Including a histogram for each. That lets me predict approximately how ...
3
votes
1answer
27 views

Parameter Inference when Model is a bad fit to the data.

I am working with gamma-ray data from the Fermi Satellite. The data has been binned into energy dependent maps of the sky -- e.g. three dimensions (energy, latitude, longitude) and is extremely high ...
0
votes
0answers
7 views

Significance of changes in the distributions of dependent samples of ordinal data

I'm taking a chance here. Would like to find out if there is some kind of statistical test that can determine the significance of changes in the distributions of dependent samples of ordinal data. I ...
6
votes
2answers
176 views

How to determine how many variables and what kind of variables a table of data has?

Here's an example regarding what I'm lost on. In the past I've just found myself counting columns to determine how many variables a table has, but I've realized that's totally insufficient. For ...
2
votes
2answers
320 views

Regression: Scatterplot with low R squared and high p-values

Based on three datasets, I have produced the scatterplot below in Python: I am trying to fit a line on each dataset, but when I check the metrics this is what I get: Set 1 (red): R squared=0.002, ...
1
vote
0answers
7 views

Statistical Data Changing Over Time

I am attempting to determine voter registration rates by race within specific precincts in my county. The local Supervisor of Elections' Office provides data online of the voter registration rates by ...
1
vote
0answers
41 views

RESET Test in R Influenced by Heteroskedasticity in the Data?

I'm running a negative binomial model in R on 558 observations of count data, along with "vcov" to add robust errors. I am using the Ramsey RESET Test as a criteria to judge my model. I have always ...
0
votes
0answers
32 views

Should 0.5 be used as a feature value in cases when it is not calculable (its range is in [0,1])?

I have a training dataset of positive and negative examples for a certain class. The problem is that, in some cases, the value of a particular feature is not calculable. So, since this feature ...
2
votes
1answer
45 views

Performing K-fold Cross-Validation: Using Same Training Set vs. Separate Validation Set

I am using the Python scikit-learn framework to build a decision tree. I am currently splitting my training data into two separate sets, one for training and the other for validation (implemented via ...
0
votes
0answers
16 views

Statistical Test for Survey Data: Count of “yes” Responses for Experimental vs Control Groups

I have a survey dataset with 20 questions. We did an experiment where half of respondents answered under one condition, and the other half under standard condition. I created a variable "YES" where ...
0
votes
1answer
29 views

finding difference between two data set [closed]

I have a data set that has 160 attributes, after I check for correlation and I used trim like ...
0
votes
0answers
18 views

A revision to a question I asked yesterday but got no response to on calculating average tenure

Here’s my question and it’s going to be pretty long winded. My company works in the online space and we sell subscriptions. We’ve been selling subscriptions since like 2011. My boss wants me to ...
1
vote
0answers
11 views

Figuring Out The Rolling Average Of Average Tenure

Here’s my question and it’s going to be pretty long winded. My company works in the online space and we sell subscriptions. We’ve been selling subscriptions since like 2011. My boss wants me to ...
1
vote
0answers
22 views

Statistical methods for adjusting heaping in birth weight data

I have a birth weight data of children under aged 5 years (sample size, n=2000). Data comes from two sources : 1. Mothers recall (n=1500) and Health card (n=500). Naturally, mothers' recall data ...
1
vote
1answer
40 views

Comparing pre and post test raw scores with one subject

I am currently in the process of trying to analyze data from research carried out. I am analyzing whether a music intervention showed improvements in rate, accuracy, fluency, and comprehension on a ...
3
votes
0answers
21 views

differences between fake computer-generated data & real data

generally, is there a/some formula(s) or algorithm(s) to prove that a data set is a fake one generated using computer and not by field work? more specific, recently I got a research questionnaire data ...
1
vote
1answer
41 views

When the regression line is not linear

I performed a regression analysis with two datasets, each of which has size 50. One dataset is called Spatial % and the other ...