Tagged Questions

1
vote
0answers
20 views

Hierachical Predictors in a Regression

Note: Mainly this question pertains to predictions from a model. If the unit of analysis of a regression (or any predictive model really) is the individual retail store and these stores are organized ...
6
votes
2answers
105 views

Is there overfitting in this modellng approach

I recently was told that the process I followed (component of a MS Thesis) could be seen as over-fitting. I am looking to get a better understanding of this and see if others agree. The objective of ...
3
votes
2answers
100 views

How I can deal with too many variables in training a data set?

I am trying to train a predictive model on whether a given person is ( male or female) based on behavior cues we've obtained from online surveys. The dependant variable will be a binary ( 1 or 0 ...
4
votes
0answers
98 views

Model performance in quantile modelling

I am using quantile regression (for example via gbm or quantreg in R) - not focusing on the median but instead an upper quantile ...
3
votes
1answer
229 views

Imputation with Random Forests

I have two questions on using random forest (specifically randomForest in R) for missing value imputation (in the predictor space). 1) How does the imputation algorithm work - specifically how and ...
7
votes
5answers
238 views

Does preclustering help to build a better predictive model?

For the task of churn modelling I was considering: Compute k clusters for the data Build k models for each cluster individually. The rationale for that is,that there is nothing to prove, that the ...
2
votes
2answers
146 views

Strange behavior of predictive model gain curve

When building predictive models (binary target), one of the principal methods I use for determining how useful the model is, is to plot the true proportion of y=1 values for each decile of the ...
2
votes
1answer
288 views

Testing for useful variables in a “net lift model”

I am often involved in modeling the Net lift, aka Uplift, aka incremental response of direct marketing campaigns. In a nutshell, this approach looks to model and thus select for marketing those ...
6
votes
0answers
158 views

Finding the correct data mining approach

(I apologise for being a newb, but I'm a researcher introducing myself to data mining---any help or insight would be greatly appreciated. Also, this isn't technically a homework question, but I've ...
3
votes
2answers
451 views

Pre-processing time series data for data mining / predictive modeling input

What are some ways to prepare/pre-process time series data to use the series data as a predictor(s) in a predictive model (classification or regression)? Specifically, what are the methods to be ...
4
votes
1answer
154 views

Incorporating a treatment into a classification scheme

I have about 400 pieces of silver of different geometric dimensions. They were assigned to six groups and each group went through a series of stress tests, such as bending, pulling, putting in fire ...
0
votes
0answers
54 views

Pre-process classification data according to ‘amount of evidence’

I have an area which is divided into polygons of different sizes. Each polygon has the same associated features/predictors and I know whether something occurs within the polygon or not. If something ...
2
votes
0answers
956 views

Does anyone have experience with IBM's “SPSS Modeler”?

SPSS Modeler seems like a great tool for data mining (especially for prediction etc.) but it is extremely costly for individuals like me (around 20,000 euros excl. tax). There is also a video. I am ...
2
votes
1answer
207 views

Figuring out probabilities with Hidden Markov Models

I'm really new to statistics so sorry in advance if this question does not make sense. Background: I'm trying to learn about hidden Markov models and they seem interesting but I was wondering about ...
10
votes
1answer
341 views

How to predict when the next event occurs, based on times of previous events?

I'm a high school student and I'm working on a computer programming project, but I don't have a lot of experience in statistics and modeling data beyond a high school statistics course so I'm kinda ...
3
votes
1answer
817 views

Bagging with oversampling for rare event predictive models

Does anyone know if the following has been described and (either way) if it sounds like a plausible method for learning a predictive model with a very unbalanced target variable? Often in CRM ...
17
votes
2answers
273 views

“Interestingness” function for StackExchange questions

I am trying to put together a data-mining package for StackExchange sites and in particular, I am stuck in trying to determine the "most interesting" questions. I would like to use the question score, ...
3
votes
2answers
310 views

How to measure/weight the importance of tags?

Lets say we have $T$ tags and $N$ articles and lets say that for each tag $t_{i}$ we know that it has tagged $n_{i}$ articles. Meaning that, frequency($t_{i}$)=$n_{i}$. Given the above information ...