Questions tagged [dataset]

Requests for datasets are off-topic on this site. Use this tag for questions concerning creating, processing, or maintaining datasets.

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Combining multiple datasets for E-Commerce data into one for modeling

I need to build an algorithm that predicts whether a customer will shop or not in the next week. There are multiple datasets for example there is a dataset that contains the items users bought, the ...
0
votes
0answers
9 views

When training data is much less than the prediction perdio

Given training data on tweets and their retweets, how would you predict the number of retweets of a given tweet after 7 days after only observing 2 days worth of data? Its strikes me that this ...
0
votes
0answers
5 views

Ranked average percentage based on data

Firstly thank you reading and secondly thanks for your answer.. if possible, I'd like to find an average in percentage format on the following data..:
0
votes
0answers
8 views

Expensive Data Acquisition — Gap finding in dataset

I am building a classification model with multiple labels. Data Acquisition is a problem because it is laborious and expensive. Therefore, I am trying to find methods to analyze the current dataset ...
2
votes
1answer
46 views

How to get the 3-digit SIC code from Datastream and merge ISIC to SIC?

When trying to replicate a paper, I faced the problem in merging two datasets. When merging an ISIC-related dataset (dataset A) to a SIC-related Datastream dataset (Dataset B), I faced a problem with ...
0
votes
0answers
7 views

are my observations are consistence

I am studying the factors that influence the den site selection by a species Hyaena. In field I observed that hyaena is only occupying rock shelters like cave and crevices but not burrow. Therefore to ...
0
votes
0answers
17 views

Split on dataset with some shared features?

I have a dataset with financial stock data, some of the features are shared, for example daily gold prices, while the stock price for each individual stock is different, the gold price would be the ...
0
votes
0answers
12 views

Listwise Learning to rank for partially labelled list of ranking data

I was reading about the Listwise approach for LTR. The first Listwise LTR paper ( refer page 3, left column, para 2-3) explicitly mentions "n(i)" which implies the number of ranked Documents ...
1
vote
0answers
21 views

Proper Test for Type of Data

Warning: I have not done any statistical analysis since 2012. Problem: I have a list of lists of lists: Each of the $289$ ($17\times 17$) element is $\{\phi_{j,1},\phi_{j,2},f(\phi_{j,1},\phi_{j,2}),\{...
0
votes
0answers
19 views

Measure to indicate dataset imbalance?

I have a dataset with 83% of positive class and 17% of negative class. While this seems to be a imbalanced dataset based on outcome class proportion and features also don't show any variabity ( to ...
3
votes
1answer
31 views

Should the median intuitively split a dataset into equally sized halves - and should two medians split into equally sized fourths?

I cannot decide if this question is patently silly or actually deep, so figured that makes it a perfect question for Cross Validated. We have a dataset with N ...
0
votes
1answer
16 views

What is the best time series model to find “time lag” between 2 sets of data?

I have two sets of data. Set 1 is a count of sales & date Set 2 is a count of event x occurring & date. What is the best method to find out how a change in an element of Set 1 (date t, sales s)...
1
vote
0answers
15 views

Best model or method for my time series data and question?

I have 2 sets of data Set 1 is composed of time, t and number of u units , (sales data) Set 2 is composed of time, t, and number of s sessions, ( count# of event s occured) Things I would like to know:...
1
vote
2answers
28 views

Are evaluation metrics computed on training dataset?

Based on my own studies and questions on this site, my understanding is that evaluation metrics (accuracy, precision, recall) are only calculated on the test dataset. The training dataset is used to ...
1
vote
1answer
13 views

What does ADE20k (the scene segmentation benchmark) stand for?

ADE20k is a scene segmentation dataset created by MIT. It is a common benchmark for localization tasks in computer vision research. I cannot find anywhere what the name stands for! This information ...
0
votes
1answer
18 views

Improvement of Interpretation of residuals vs fitted plot

I wish to examine assumptions for single regression using the built in diagnostics within R. The residuals vs fitted plot below seemly violated the assumption. There are 2 dummy coded variables within ...
-1
votes
0answers
17 views

How do we produce synergetic effect from three data sets?

Let’s assume, for a particular imaging system; we have three different sets of data – D1, D2, and D3. By different, we mean the event corresponding to one data set is different to the other. But all ...
0
votes
1answer
32 views

how can i use a frequency chart and a density curve to check for normal distribution in my data

I've seen in many analysis reports that to test for normality in a discrete variable, as well as in a continuous variable, people use a frequency chart and a density plot to visually check for ...
0
votes
1answer
16 views

Which of these two experiment designs converges faster?

This is an abstract question, an attempt at roughly heuristicking an answer that would otherwise take significant cost and data to answer. If the "fuzziness" of the question makes it a bad ...
0
votes
1answer
14 views

Generating data for one-class dataset

In case I have a dataset that have only one class unlabeled (benign), could you please list some algorithms/papers that are used to generate complementary data (malignant) based on benign data only? I ...
0
votes
0answers
6 views

Constructing a dev set for multi-class classification

My total data available is close to 400,000 observations. Total of 450 labels. Label distribution is imbalanced, which mirrors what happens in the wild. Because the labels occur in the wild in the ...
0
votes
0answers
6 views

Standardized image intensity distribution

I plotted a channel-wise histogram of my dataset after subtracting its mean and dividing by the std, thus normalizing it. It looks like this. Calculcating mean and std again confirms that it has ...
0
votes
0answers
5 views

How to assess reliability and validity of questionnaire compared to existing questionnaires?

I have a sizeable data set (e.g., 200+) of responses on ordinal scales (e.g., 1-4) for several questionnaires. One of these is newly designed by me. I conducted several questionnaires with the ...
0
votes
0answers
8 views

Would an SVM be more successful with binned attributes or non-binned attributes?

Working on an SVM through WEKA and testing which kernel would operate better. I have some data values which are nominal, but about half of them are numerical values with all unique values. There's ...
0
votes
1answer
22 views

Data preparation in cross-validation scenario

In a k-fold cross-validation scenario for validating a regression model, how/when should I do preprocessing of the data? I'd like to know especially when to remove outliers, empty columns etc, i.e. if ...
0
votes
0answers
7 views

How to reconcile different data providers when doing panel econometrics?

Imagine you have several panels of (time, entity, variable), recorded by several observers and hence different in sparsity and variable values. How does one efficiently use all of them for estimation? ...
1
vote
0answers
27 views

How to create a correlation matrix in a dataset with multiple datatypes?

I am stuck in a peculiar problem where I need to know the correlation between attributes. So far it has been easier, where all of my input attributes used to be only numeric, and hence without any ...
1
vote
1answer
12 views

The accuracy in ANN model's training set and validation set keep incerasing synchronistically [closed]

I have two different datasets: A and B. I separated dataset A into A1 as a training set and A2 as validations set. However, I find the accuracy of the two datasets keeps increasing when I train the ...
0
votes
1answer
17 views

question about standardizing data

Ok so I'm confused about the whole concept of standardizing data. I get the concept of why we need to standardize data for, let's say multiple linear regression so the data points are similar, but ...
0
votes
1answer
20 views

what is the recommended smallest value of AIC or BIC? [duplicate]

to now how small it is! in addition what value of AIC and BIC are the minimum value it get manly for categorical data analysis
0
votes
0answers
8 views

How to construct a dependent variable when the focus is on the first instance of occurrence?

I am constructing a time-series dataset on disease modeling to forecast the first instance of a disease occurrence or forecast disease risk as a function of past conditions. I plan to construct a ...
0
votes
1answer
33 views

Is using the mean and the variance independent of probability distributions? [duplicate]

I want to know if using the mean and the variance is independent of a probability distribution or not. I just want to make sure that whether using the mean and the variance is independent of ...
2
votes
1answer
43 views

How to measure a spike in data

For example, if I had data like the number of daily coronavirus cases for different countries and I wanted to compare the 'intensity' of the spikes (abnormally large number of cases), what statistical ...
0
votes
0answers
9 views

How to interpret this dataset: Symbol Detection for Molecular Communication

I am going to participate to my first data competition, and I have lot of doubts, that I hope you can help me to clarify. This is the background: Detecting the transmitted symbols is challenging in ...
1
vote
0answers
9 views

Multiple Regression Extreme Values in all Covariates

In multiple regression: If we have a data set with covariates having extreme values, the max value of the covariate is around 10 to 14 times the mean value of that covariate. This is occurring for all ...
0
votes
0answers
11 views

How to know if we can find a separating hyperplane?

Given a data set, how do we know if it's possible to find a separating hyperplane? Do we need to know infomation like if the data set is separable or not, or is it enough to know the number of ...
1
vote
1answer
97 views

How to sample from different datasets such that they have similar distributions?

I have data from multiple datasets with the boxplot given below In the above figure, I have data from 7 different datasets. I am looking for a sampling strategy without replacement such that samples ...
0
votes
0answers
23 views

Dimensionality Reduction on Mixed Data

If you have a dataset with not only numerical data, but also categorical and boolean. How do you approach doing dimensionality reduction to this dataset. The standard algorithms only work with ...
0
votes
1answer
25 views

what type of data analysis should I carry out

In my experiment I have two groups each with 30 participants (experimental group and a control group) The experimental group consists of autistic individuals and the control consists of neurotypical ...
0
votes
0answers
11 views

How should I create rating matrix for implicit data?

My task is very simillar to kaggle: Instacart Market Basket Analysis. I need to predict which products will be in a user’s next order. So i have data wich looks like this Now am working on building ...
0
votes
0answers
10 views

Calculating spread (standard deviation or CV) for ranges of values

Due to data suppression/confidentiality, I only have ranges of values, not individual data points. I have the sample sizes, the percentage of acres that receive ranges of fertilizer application rates, ...
0
votes
0answers
11 views

Statistical data analysis [duplicate]

I'm currently facing some issue in determining what statistical test might be best to adopt to determine whether or not three datasets are significantly different from each other. I have the following ...
-2
votes
1answer
43 views

Statistical data analysis - data comparison [closed]

I'm currently facing some issue in determining what statistical test might be best to adopt to determine whether or not my three datasets are significantly different from each other. I have the ...
0
votes
0answers
13 views

Not all independent variables available for same time period..how to handle such situations for ML models?

I have 9/10 independent variables for which information is available for last 2 years (including dependent variable). For 1/10 variables the information is only available for last 2 months as this ...
0
votes
1answer
34 views

this is not overfitting but something else, right?

I trained a VAE on a dataset containing 1k images. The VAE itself is convolutional, downsamples 256x256 rgb images four times before reconstruction and uses both relu and BatchNorm Layers as well as ...
0
votes
0answers
6 views

How to quantify different sizes and colors of pie slices?

Suppose want to build a regression model that compares pies (say, "goodness scores"). One of the properties of these pies is that they are cut into differently sized slices across pies and ...
4
votes
3answers
617 views

What does it mean when a Data Matrix has full rank?

Suppose I have a matrix of observations $X \in \mathbb{R}^{n \times p}$. So I have $n$ observations that each have dimension $p$. Lets suppose also that we have more data points than dimensions to ...
0
votes
2answers
27 views

How much data do I need to build a classification model?

While studying a machine learning based classification model, I started wondering how much I need to build a classification model. The way I do is that I download some specific dataset within a ...
0
votes
0answers
15 views

Separating hyperplane and soft-margin classifier

I have some questions that I'm supposed to answer, but I have no idea where to begin. Any hints or solutions would be helpful. I feel like I don't have enough information about the dataset to answer ...
1
vote
0answers
9 views

Statistical Responses/tests for Yes/No, Pre/Post surveys

I am completing a research paper regarding clinician adherence to a protocol. One of the questions is whether or not they use adjuctive therapy to the treatment. I have a pre and post survey, yes/no ...

1
2 3 4 5
35