Questions tagged [data-mining]

Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining of association rules.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Patient Services Probabilities Prediction

I have a dataset in which I have a patient's diagnosis and service corresponding to those diagnoses. I want to apply machine learning in such a way that. When I enter the patient diagnosis to system ...
0
votes
0answers
8 views

How to compare the behaviour of different pretrained RL models?

I have a number of pretrained RL models (PPO2, ACER, ACKTR, ...) and I want to compare their behaviour in the environment. This includes their performance in respect of the episode-reward as well as ...
0
votes
0answers
10 views

Dession tree induction [closed]

If I have three attribute two have same gain which is our root node in dission tree inductionI share picture some one help me
0
votes
1answer
37 views

Yeo-Johnson does not increase normality

I have used Box-Cox Yeo-Johnson transformation to make my skewed data columns less skewed and more normal so that I can remove outliers. e.g. originally most of my columns have a 'skewness' of 400! ...
0
votes
0answers
48 views

removing outliers in skewed data for xgboost

i have a couple of columns in my data which are postively skewed. they are non-normal from the hist plots. plotting a qq plot further cinfirms this. i should remove outliers from my data for xgboost. ...
1
vote
0answers
16 views

Is K-medoids / partitioning around medoids (PAM) appropriate for clustering data with many zero values?

I need to cluster a matrix which contains zero values. I am clustering three separate sets of 24 values. The first two are non-zero and represent hourly ambient temperature (in K) and electrical ...
0
votes
0answers
46 views

Which machine learning algorithm is best to use for prediction/ranking in this dataset?

Each row in the training set depicts co-occurring symbols (one training sample). The sequence doesn't matter, only the co-occurrence does. a, b, d, e b, c, e, z z, a b, f, g, i, s, u .... .... ...
1
vote
1answer
28 views

Interpreting the results of p-value and Cramer's V value

For my assignment I am working on a data set with the sample size n = 4.000.000 and about 450 columns. I have 4 rows in my cross table that have the values winter, spring, summer and fall. So my cross ...
0
votes
0answers
7 views

why k-means is better in clustering than topic modelling algorithms like LDA?

I want to know about the advantages of K-means in clustering essays to discover their topics. There are a lot of algorithms to do it such as K-medoid, x-means, LDA, LSA, etc. Please give me a full ...
1
vote
0answers
13 views

Mahalanobis distance between high dimensional arrays

As we know, the Mahalanobis distance (MD) is one of the distance metrics for measuring two points in multivariate space. In practice, I can compute Mahalanobis distance between two 1D arrays using ...
22
votes
6answers
2k views

What are the disadvantages of using mean for missing values?

I have an assignment (Data Mining course) and there is a part which asks: "What are the disadvantages of using mean for missing values?" in Missing Value section. ...
0
votes
0answers
10 views

new features selection

I am in a project in which I have a specific description of a certain binary profile for which I have about 200 positive examples and another 200 negative. This description is given from about 60 ...
0
votes
0answers
12 views

Stacking model: is finding best threshold necessary?

After performing a stacking model (with rstudio), is it necessary to choose the best threshold for it? In general after finding the best model among all the fitted models , you have to choose the ...
1
vote
1answer
24 views

How to measure smoothness of inputs over outputs?

I know similar questions have been asked for time series data. But my question is a little bit different. Consider that we have input dataset $X \in R^{N \times M}$, where $M$ is the dimension of ...
0
votes
0answers
8 views

Different normalization techniques (mean and min-max) on different columns in a data frame

Is it possible to have different normalization techniques (mean and min-max) applied on different columns in a data frame?
0
votes
0answers
15 views

Predict for an outcome within a time window

I have a dataset which has around 10K records. My objective is to predict whether the customer will churn or not. Binary classification problem with each class representing around 55:45 proportion ...
0
votes
0answers
14 views

Compute correlation between pairs of attributes for each class label

I am working on collecting some dataset characteristics for binary classification tasks, and I want to calculate the correlation following the measure proposed by this quote: The correlations ...
0
votes
0answers
12 views

What approach or unsupervised methods can be used to pick out patterns in noise?

This is a hypothetical situation. Let's say you have access to a lot of human behaviors and characteristics (features). Let's say you have a sample of 10000 humans. You know that within this sample, ...
0
votes
0answers
29 views

Low sample size with independent observations

I am looking at sports team level data (summarized by average in each season) over several seasons and would like to predict/classify the winner of the championship. In a single season, the data has ...
1
vote
0answers
25 views

How is domain knowledge developed?

Most sources I've read state that domain knowledge is crucial for making good inferences. As an example, if I'm conducting a study to assess the importance of a new biomarker for heart attacks, the ...
0
votes
0answers
7 views

Predicting 3-value class label with binary decision tree

Is binary decision tree working better while predicting class label that is binary as well (2 possible values) or it doesn't have any impact? If it does, would it be a better idea to use ternary ...
0
votes
0answers
12 views

Cuboids and cells

Hi I'm new to data mining, I have some questions about the relationship between cells and cuboids in data cube. Are they basically the same thing? How many cells can a base cuboid have? How many cells ...
0
votes
0answers
29 views

R - Making legit RandomForest results reproducible with set.seed

I guess that my question is kinda weird but: I'm working on a university project where I have to use a RandomForest model to predict if patients have depressive tendencies. And while I'm getting ...
2
votes
2answers
71 views

What does the true level of significance mean in data mining?

There is a formula α*=1-(1-α)^c/k, a* - true level of significance. a - nominal level of significance. c - the number of candidate regressors. k - the number of finally selected regressors. I ...
2
votes
1answer
36 views

How to select best feature set?

I am currently using feature selection approaches like filter, wrapper, embedded etc. All these methods give different set of features and I rank them based on their frequency of occurrence in other ...
1
vote
2answers
293 views

How to adjust confounders in Logistic regression?

I have a binary classification problem where I apply logistic regression. I have a set of features that are found significant. But I understand that Logistic regression doesn't consider feature ...
1
vote
1answer
24 views

How to split training data when learning DNN for unknown test data?

I'm designing a CNN model for a data mining competition in which we are provided with N sample of training data. We do not know the test size, but presumably it is from the same distribution as ...
2
votes
2answers
338 views

One hot encoding of a binary feature when using XGBoost

I already asked this question is SO; however, I realized that this may be a better place for this type of question. I am well aware that when using categorical features with tree based models such as ...
0
votes
1answer
130 views

Number of Parameters to be learned in k Guassian Mixture model?

How many parameters do we need to learn for a k-Gaussian mixture model, where all mixtures have the same spherical radius?
1
vote
2answers
62 views

Event Identification in Series

I've just posted this question on Data Science SE which asks about machine learning methods to identify "events" in series (time-series or otherwise). I'm wondering if I should consider regressions ...
0
votes
0answers
22 views

How to identify important features in deep learning models

I am mostly familiar with traditional hand-crafted feature setting where we use ML algorithm such as SVM to analyse these features. In this way, we can identify what were the most important features ...
0
votes
0answers
25 views

Stratification of sample data is lowering my accuracy?

So I've got this trainingset, it has a bunch of stuff yada yada.. Main point is that there are two target variables that only occur once in the dataset. This means I can't stratify when sampling, I ...
0
votes
1answer
38 views

Data Mining/Statistical Methods to find trends, peaks, etc

currently I am working on a project for my final exam. The data is coming from a streaming plattform. The data I am using are some logging data (data when customers have problems with the streaming ...
0
votes
0answers
23 views

Statistical error in the approximation-estimation tradeoff

Show that $$E(g_\tau ^G(X)-g^* (X))^2 = E(X^T \hat{\beta}-X^T\beta^G)^2+E(X^T\beta^G-g^*(X))^2$$ where $g_\tau ^G(X) = X^T \hat{\beta}$ and $g^G(X) = X^T \beta^G$ where G is a class of linear ...
0
votes
1answer
18 views

Model Integration

I have two models one of them made by using a large dataset, the other one made by using a small dataset. I want to integrate these two datasets. The large dataset has more samples of rare instances. ...
0
votes
1answer
186 views

Why is the correlation of a variable and itself a histogram?

This post is visualizing the Wine dataset. You may have noticed that the figures along the diagonal look different. They are histograms of values of individual variables. We can see that the "Ash" ...
0
votes
1answer
108 views

What an input space exactly is in the context of machine learning?

I've been confused about various "space"s in machine learning for a long time. I've checked out this, this and this post. I am trying to get understanding through some concrete examples like this ...
1
vote
2answers
87 views

Should you take a sample when doing EDA?

Suppose i have a large dataset, such that python graphing libraries are unable to handle. Is it a good idea to take a random sample? Specifically if it's a classification task, and where the target is ...
0
votes
1answer
20 views

Can someone give a concrete example of exploitation in the context of Exploratory Data Analysis?

This post says Exploratory Data Analysis (EDA) consists of 2 steps exploration and exploitation. I know a little about exploration which uses some techniques such as data visualization to understand ...
0
votes
1answer
88 views

Interpret the clustering results of Weka to measure the performance [closed]

I'm having the Boston dataset, where it's class variable in the housing price. So I think regression is more suitable for this dataset, so we can predictions. I'm using Weka for this. I used several ...
0
votes
0answers
31 views

What are the state of the art algorithms for association rule mining?

I have done some work with the apriori algorithm to mine for association rules in a market basket dataset. However, the number of rules returned is enormous and a large part of the rules are just some ...
0
votes
0answers
33 views

Can anyone suggest the source to study basics needed for The Elements of Statistical Learning by Trevor Hastie Robert Tibshirani Jerome Friedman [duplicate]

anyone suggest the relevant study material to get the basics cleared for understanding the book The Elements of Statistical Learning by Trevor Hastie Robert Tibshirani Jerome Friedman
0
votes
1answer
74 views

What is first order difference in trend analysis?

I am following this paper: Measuring, Predicting and Visualizing Short-Term Change in Word Representation and Usage in VKontakte Social Network where in Differencing Statistics section they describe ...
1
vote
1answer
481 views

How to calculate tf-idf for a single term

I am following the tf-idf method described in this paper: Measuring, Predicting and Visualizing Short-Term Change in Word Representation and Usage in VKontakte ...
1
vote
0answers
9 views

How to identify important categories in categorical variable to predict some binary variable?

Imagine that we have two sets of data. The first one contains information about websites visited by certain users: ...
0
votes
0answers
13 views

In a particular KNN with a particular dataset, what exactly the hypothesis space is?

Wiki gives this setting for KNN Suppose we have pairs ${\displaystyle (X_{1},Y_{1}),(X_{2},Y_{2}),\dots ,(X_{n},Y_{n})}$ taking values in ${\mathbb {R}}^{d}\times \{1,2\}$, where Y is the ...
1
vote
1answer
168 views

scikit-learn feature selection on k-fold loop

I am using the iterator of StratifiedKFold from sklearn and i've noticed that i must include a process of feature selection on ...
3
votes
2answers
171 views

Could anyone explain the terms “Hypothesis space” “sample space” “parameter space” "feature space in machine learning with one concrete example?

I am confused with these machine learning terms, and trying to distinguish them with one concrete example. for instance, use logistic regression to classify a bunch of cat images. assume there are 1,...
2
votes
1answer
222 views

Why is bagging stable classifiers not a good idea?

Citing Bagging Predictors, Section 4 (emphasis mine): Let each $(y, \mathbf{x})$ case in $\mathcal{L}$ be independently drawn from the distribution $P$. Suppose $y$ is numerical and $\phi(\mathbf{x}...
1
vote
1answer
127 views

Choosing the Right Data Mining Classification Technique

From the different Data Mining tasks, I want to train the Classification. For that, I: Took this dataset (can be used as an example for the answer). Got to know the data (data objects, attribute ...

1
2 3 4 5
23