Questions tagged [data-mining]

Data mining uses methods from artificial intelligence in a database context to discover previously unknown patterns. As such, the methods are usually unsupervised. It is closely related but not identical to machine learning. Key tasks of data-mining are cluster analysis, outlier detection and mining of association rules.

Filter by
Sorted by
Tagged with
6
votes
2answers
354 views

Is there a bias correction for effect size in a data mining context?

Given $K$ possible 'treatments' of some kind, and independent observations of some response under those treatments, say $X_{i,k}$ for $i=1,\ldots,n_k$ and $k=1,\ldots,K$, I am faced with the classical ...
6
votes
4answers
14k views

Reducing false positive rate

I am running a model for a problem in insurance domain. The final results show some false positive x and some false negative y. I am using SAS Enterprise Miner for this. Can somebody suggest me how to ...
3
votes
1answer
198 views

Extrapolating the amount of data missing from the amount of data partially missing

Suppose I have a fleet of two-car trains riding around, and that each car is equipped with a data recording device. Unfortunately, some of the recording devices aren't working. I don't know either the ...
2
votes
3answers
241 views

Methods of grouping sets of data

I have data which has several properties (metadata, as key value pairs, where the keyspace is shared over the whole dataset) per object. I took a sample of objects and divided them in n groups ...
6
votes
2answers
289 views

Discerning between two different linear regression models in one sample

Suppose I observe a sample $(y_i,x_i)$, $i=1,...,n$. Suppose that I know the following: $y_i=\alpha_0+\alpha_1x_i+\varepsilon_i$, $i \in J\subset\{1,...,n\}$ $y_i=\beta_0+\beta_1x_i+\varepsilon_i$, ...
218
votes
13answers
200k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
4
votes
2answers
407 views

How to measure/weight the importance of tags?

Lets say we have $T$ tags and $N$ articles and lets say that for each tag $t_{i}$ we know that it has tagged $n_{i}$ articles. Meaning that, frequency($t_{i}$)=$n_{i}$. Given the above information ...
6
votes
1answer
764 views

Data mining algorithm suggestion

I would like to use data mining to try to find a good workout schemes. The input dataset will contain the parameters of a set of workouts with dates and different performance and medical measures. The ...
27
votes
9answers
16k views

Statistics and data mining software tools for dealing with large datasets

Currently I have to analyze approximately 20M records and build prediction models. So far I have tried out Statistica, SPSS, RapidMiner and R. Among these Statistica seems to be most suitable to deal ...
14
votes
3answers
13k views

Flowcharts to help selecting the proper analysis technique and test

As someone who needs statistical knowledge but is not a formally trained statistician, I'd find it helpful to have a flowchart (or some kind of decision tree) to help me choose the correct approach to ...
16
votes
10answers
1k views

A survey of data-mining software tools

Although I was trained as an engineer, I find that I'm becoming more interested in data mining. Right now I'm trying to investigate the field further. In particular, I would like to understand the ...
8
votes
10answers
564 views

Data mining conferences? [closed]

What are the most significant annual Data Mining conferences? Rules: One conference per answer Include a link to the conference
21
votes
6answers
16k views

What is the difference between data mining and statistical analysis?

What is the difference between data mining and statistical analysis? For some background, my statistical education has been, I think, rather traditional. A specific question is posited, research is ...
6
votes
2answers
427 views

How to limit my input data for Jaccard item-item similarity calculation?

I'm trying to compute item-item similarity using Jaccard (specifically Tanimoto) on a large list of data in the format (userid, itemid) An item is considered as ...
11
votes
2answers
3k views

What is the difference between functional data analysis and high dimensional data analysis

There are a lot of references in the statistic literature to "functional data" (i.e. data that are curves), and in parallel, to "high dimensional data" (i.e. when data are high dimensional vectors). ...
13
votes
4answers
6k views

Best ways to aggregate and analyze data

Having just recently started teaching myself Machine Learning and Data Analysis I'm finding myself hitting a brick wall on the need for creating and querying large sets of data. I would like to take ...
10
votes
6answers
865 views

Dubious use of signal processing principles to identify a trend

I am proposing to try and find a trend in some very noisy long term data. The data is basically weekly measurements of something which moved about 5mm over a period of about 8 months. The data is to ...
3
votes
2answers
2k views

What are the key statistical concepts that relate to data mining?

What topics in statistics are most useful/relevant to data mining?
7
votes
3answers
1k views

Data Mining— How to Tell Whether the Pattern Extracted is Meaningful?

I am sure that everyone who's trying to find patterns in historical stock market data or betting history would like to know about this. Given a huge sets of data, and thousands of random variables ...
11
votes
6answers
3k views

Algorithms and methods for attribute/feature selection?

I have data with continuous class and I'm searching for good methods to reduce number of attributes. Now I'm using correlation based filters, random forests and Gram–Schmidt algorithm. What I want ...

1
20 21 22 23
24