Questions tagged [dataset]

Requests for datasets are off-topic on this site. Use this tag for questions concerning creating, processing, or maintaining datasets.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Which is the best online data science specialisation course? [closed]

I want to switch my career from Marketing to data scientist and I'm looking for the best institute. Also, I would want to know if switching the career is a better option or not?
2
votes
1answer
49 views

Is it reasonable to set the null hypothesis as the well-accept classic model?

Suppose that I did an experiment and collected a dataset. In previous literature these type of dataset are explained by a well-accepted model, called Model M. I believe that the Model M cannot explain ...
1
vote
3answers
137 views

Datasets that "simple" models fail but more complex models work

Notes: The term "simple" in this context is defined at the end of the question; I believe this question is somehow not about dataset request even if its title is named this way... Main ...
0
votes
0answers
19 views

Data Science Forums in the UK [closed]

I was just wandering if anybody knows of any UK-orientated groups/forums out there (slack/teams/discord/web/facebook etc) for those who are learning data science/analysis? I understand there are ...
0
votes
0answers
10 views

Scaling columns pandas DataFrame

For a dataset having different numeric columns, they usually have different range and distributions. As an example, I have used the Iris dataset. The distributions of it's 4 columns are shown: My ...
0
votes
0answers
12 views

Can we consider a dataset representing a particular use case, a sample? [closed]

We find many datasets on various online websites. Can we consider a dataset as a sample containing fixed number of rows, where each row can be considered as a data point?
0
votes
0answers
25 views

Confusion between ANOVA and MANOVA

I have a dataset with participants which have used two methods (method 1 & method 2) to measure a variable with different levels (5 levels) in minutes. My intention is to see whether there are ...
0
votes
1answer
20 views

How to train on one dataset and deploy on another without loss of performance?

This could be a general question but something that concerns ml4health more. When you train your ML model using one dataset collected from one centre and would like to deploy your model on the data ...
0
votes
0answers
7 views

Generalised linear modelling data layout

I am working with choice test data from an Ecological study. I have multiple tests, where individuals were given the choice in each experiment to choose one of two options (control or treatment/ 0 or ...
1
vote
2answers
28 views

What would be a reason to use the Root Mean Square Error (RMSE) to combine data?

In this machine learning tutorial by Google they use the Root Mean Square to create a similarity measure between two shoes. They first calculate the difference between the size of two shoes and then ...
0
votes
0answers
6 views

Inferring the coefficients of two datasets by analyzing the coefficients of the combined dataset

I have two datasets, but for some reason, and in each of them I need to do a regression analysis of Y = aX + bZ +e. Assume for some reason I cannot do the regression analysis for each dataset, but ...
0
votes
0answers
12 views

What are "instances" and what are "attributes" in time-series dataset?

I have a table with 370 entitled columns and 140256 time-titled rows in ElectricityLoadDiagrams20112014 Data Set. Nevertheless, home page says ...
0
votes
0answers
16 views

Opportunity Evaluation within excel

I'm a student of data analytics. I've the following raw data https://1drv.ms/x/s!Amd7BXzYs7AVhAqQGtBZLHzQ1xZ6 The main goal as for finished example :https://1drv.ms/x/s!Amd7BXzYs7AVhAsWa0e5yvqPhuRf?e=...
0
votes
1answer
19 views

How are R-square and mean absolute error related? Which is more important for evaluating performance?

I have some data which is divided into training and test. I ran regression using training data and got an R-square value of 0.8, and a mean square error of 10 using the original and predicted values ...
0
votes
0answers
22 views

Explained variance for training dataset

After computing the unbiased variance of each attribute in my dataset, I am now interested in finding the explained variance / attribute. I have not trained a model, and therefore I am simply looking ...
1
vote
1answer
19 views

Pre-processing of features with cross-validation

I want to determine the best values of the hyperparameters of my model. In that respect, I am using a 5-fold CV approach + a test set defined as unseen data. That is, you have 5 folds: Fold 1: ...
0
votes
0answers
17 views

Testing multivariate data for normality (in R)

I have data that I want to run an ANOVA on, but I need to test it for normality. Do I test the whole dataset for normality or each subset of the data associated with a unique group level? For example ...
0
votes
0answers
8 views

Joint Model predictions without data stored

I've produced a Shiny app which allows users to run Joint Model predictions using a model object JM created by the JM package in ...
2
votes
1answer
38 views

How to combine response data of different types for modelling fruit crop size?

Some context first: For a section of my research project I am investigating how environment (soil, climate) influence the number of fruit (crop size) produced by individual trees of the same species ...
0
votes
0answers
7 views

What are general guidelines to construct a unique measure of similarity for your data?

I'm taking an introductory course in data science. As part of an assignment I'm given a dataset where I must conceive a new measure of similarity (no cosine similarity, or euclidean distances allowed, ...
0
votes
0answers
20 views

Pose Classification: how to cover non classified poses?

I'm currently working on a program to detect if a person is using a phone (either by making a phone call or by just typing). I'm planning on doing it by using a combination of phone detection via YOLO ...
0
votes
1answer
18 views

Can the number of Target features (y) exceed the number of Input features (X)?

I am trying to perform a train_test_split() on a dataset. Before doing so and while I am assigning data to X and y variables, I realized I have 8 Input features a.k.a. Independent variables and 45 ...
2
votes
1answer
21 views

Kruskal-Walis interpretation with too many outliers and "0" values

I want to establish significant differences between certain groups. I have a dependent variable that is named "NRPS BGCs" which are the presence of certain genes founded in certain bacterial ...
0
votes
0answers
7 views

Document ML model's features

I'm searching for a good way to document ML model's features. For example: The first column can be: Feature name Second column: Feature type (int, string and etc) Third column: Feature description Of ...
1
vote
1answer
18 views

Will training a classifier on data where one class is more present than another going to mess everything up?

Let's say I have a dataset where we have examples belonging to either class A or class B but we have more examples belonging to class B than A. If I train a classifier on this data will it be useless?
1
vote
1answer
34 views

Removing instances that decrease accuracy of Machine learning algorithm Methodology

Is it bad practice to run a Machine learning algorithm on an experimental dataset, check the MAE, and remove the instances that have a value of MAE above a certain limit? If we run the algorithm ...
1
vote
0answers
18 views

Best way to train a model having train/test/val set

I'm new in the machine learning field and I'm trying to understand what are the best practices to manipulate the time series. I currently have a dataset structured as well: X0: the first part of my ...
0
votes
0answers
29 views

Which statistical test to use? T-Test, ANOVA, Linear Regression?

We are conducting a research study and are stuck on the statistics portion. Hence we are looking for feedback. Some details on the experiment below. Two groups: Intervention group Control group ...
0
votes
0answers
18 views

Doubts about generating a synthetic dataset according to a paper

I'm trying to replicate the experiment reported in section 3.3 of this paper (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2930825/) but I'm struggling to understand how the synthetic dataset is ...
0
votes
0answers
8 views

Making predictions from regressors trained on two data sources

Suppose that I have data from 2 sensors (A and B) measuring the same quantity, ex. amount of precipitation. I do not know which of the 2 sensors is more accurate. I construct a regression model using ...
1
vote
1answer
16 views

When augmenting data, shall the dataset keep a balanced ratio

When training a model it is more and more common to augment data. posts indicate that only the training set shall be augmented. On the other hand it is common to split dataset in a fashion following ...
0
votes
0answers
22 views

Unsure how to analyze count data. Plot the logarithm?

My experiment consisted in turning off multiple genes separately from each other and counting a certain type of cells (say cells A). For each gene I had around 20 organs for which I counted cells A. ...
3
votes
1answer
45 views

Need data preparation cheatsheet / guidelines / first principles to train team members! [closed]

Really ran out of ideas and hence such a basic question to the community - Despite the repeated emphasis on ensuring data accuracy/validity, team members just do not spend enough time on it because ...
0
votes
0answers
47 views

Is it valid to remove trials as outliers using IQR?

I have a repeated measures experiment where all participants completed several trials for each condition. My dependent variables are response time and accuracy. I am using the Interquartile Range as ...
0
votes
0answers
16 views

Is there any rule about the ideal ratio between the number of samples and attributes for proper regression model building?

I am working on a regression problem and for that I am removing the identified outliers. I am curious to know if I should try to maintain a ratio between no. of samples and no. of attributes while ...
0
votes
0answers
12 views

Introducing correction based on intersection of two datasets and usage of the resulting values

Imagine having a set of samples, f.e. id# 1 - 10. Samples 1-7 were analyzed by some trusted method, and samples 4-10 - by a less trusted technique. These methods give different results on a subset of ...
3
votes
2answers
130 views

How to reduce RMS error value in regression analysis & predictions - feature engineering, model selection

There's this dataset containing the metadata of Twitch's top 1,000 streamers of 2020. You can have the details here. I am currently participating in a challenge to predict the values for Followers ...
1
vote
0answers
16 views

Divide training data into multiple group, and train seperately

If I have a training data, I divide them onto many group. Then, for each group, I use a different neural network to train seperately/independently. Do you think this will be better than using a single ...
1
vote
0answers
24 views

How to circumvent the Error: vector memory exhausted (limit reached?) when saving large Tidymodels workflowsets tibble [closed]

I have a question related to the methodology of working with tibbles and tidymodels workflowsets. I have a large number of models fitted and tuned using several recipe pre-processing steps/feature ...
1
vote
1answer
24 views

Multilevel multinominal logistic regression

For the analysis of repeated measurements data by developing a multilevel multinominal logistic model, in what format should the data set be prepared? What about any other technical pearls?
1
vote
2answers
38 views

Time series Algorithm suggestion

I have a univariate timeseries data which is always cumulative. So the trend is always upwards. I want to build timeseries forecast for future forecast. Currently i have 5 different models running: ...
1
vote
1answer
30 views

How to combine two or more columns which are giving same information?

I am trying to predict the closing price of a cryptocurrency. As its price depends on lots of factors, so I have lots of columns. But a bunch of columns in my dataset are kind of adding the same set ...
0
votes
0answers
4 views

What to do when there is no cointegration in dynamic panel data analysis?

I have tested for Pairwise correlation (confirmed no multicollinearity) cross-section dependency test (found cross-sectional dependence among the variables) unit root test (variables are both I(0) ...
0
votes
0answers
16 views

Is weighted linear regression best way to do this?

I have an array of 316 points (from a map) that have 64 values (various data from the map) that describe each of those points. I will then get 1000+ points from a region of the map that will also have ...
0
votes
0answers
26 views

When and how to transform data in regression tasks [duplicate]

I'm studying linear regression tasks and some transformation of data, in order to make the model linear, and to make the L-I-N-E assumption of linear regression met. I know that if we have a non-...
0
votes
0answers
18 views

Quantitative difference of two datasets

I have two biological datasets with different samples (rows) and same features (columns). I mean that I have D1 (M1 x N) size and D2 (M2 x N) size. M1, M2 is order of 2000, N is order of 500. All ...
1
vote
1answer
24 views

Does time increase linearly relative to data size in a machine learning workflow?

Let's say I am working with a dataset first containing 100 rows of data. I want to follow the normal process of doing some feature engineering, then training a model on this data. Running the whole ...
1
vote
0answers
14 views

Predicting the lifetime of events using ML, based on observation window

I am trying to predict the lifetime of events based on the features X of the event. There are some caveats to the samples that make training the NN non-trivial: Samples were gathered in a small ...
0
votes
0answers
7 views

Long format data coding for linear mixed model

Hi:) I have questions about the linear mixed model. I need to make coding for linear mixed model analysis. I have two groups (control vs experimental), pre/post/follow-up scores. In that case, should ...

1
2 3 4 5
36