Predictive models are statistical models whose primary purpose is to predict other observations of a system optimally, as opposed to models whose purpose is to test a particular hypothesis or explain a phenomenon mechanistically. As such, predictive models place less emphasis on interpretability and ...

learn more… | top users | synonyms

0
votes
1answer
12 views

using non-cancerous patients to predict survival time from cancer?

I have data that consists of lab test results for patients, cancer dates for those who got cancer, and time till death/censor from that cancer. This cancer is both rare (most of the patients never get ...
1
vote
2answers
40 views

Interpretation of LASSO regression coefficients

I'm currently working on building a predictive model for a binary outcome on a dataset with ~300 variables and 800 observations. I've read much on this site about the problems associated with stepwise ...
0
votes
0answers
9 views

Event prediction through path analysis?

I have the following problem: I want to predict an event's occurrence by investigating the steps a user goes through. Fe. I want to analyse which webpages a user visited that lead him to buy or not ...
4
votes
1answer
31 views

Where is there bias-variance trade-off, and why?

In Wikipedia, the "Bias–variance tradeoff" is mentioned in the context of prediction models where one can control the complexity of the model with some tuning parameters, and the more complex the ...
2
votes
2answers
102 views

P>0.5 cutoff not “optimal” for logistic regression

PREFACE: I don't care about the merits of using a cutoff or not, or how one should choose a cutoff. My question is purely mathematical and due to curiosity. Logistic regression models the posterior ...
0
votes
0answers
24 views

Time Series Prediction using Machine Learning

I am trying to predict the request arrive time of some objects in network traffic. I have few features of the object like their type, size, previous arrival time, etc. So I was think that I should use ...
1
vote
0answers
16 views

Dealing with correlated predictors when using LASSO

I am developing a prognostic index using the LASSO technique and wondering how to deal with the highly correlated predictor variables. Should I choose the ones I want to include in the LASSO a priori ...
1
vote
1answer
23 views

Prediction interval vs. confidence interval in linear regression analysis

I know such a problem is explained many times, but I have still a problem with the concept and interpretation: I would like to estimate export weight for 2016 The red point is estimated point red ...
0
votes
0answers
24 views

Interaction between 2 quadratic variables

If I have a regression y ~ x1^2 + x1 + x2^2 + x2 + bias, and I want to include interaction between the two quadratic variables, do I make the new regression y ~ x1^2 + x1 + x2^2 + x2 + x1^2*x2^2 + x1^...
0
votes
1answer
12 views

Prediction of next event occurrence

I am working on a request prediction problem in which I have to predict which object will be requested in the near future and how many times. This is like a basic internet traffic request pattern on a ...
1
vote
1answer
31 views

Tune parameters from a specific equation in R

This is the first time I am truing to tune model parameters in R. I have a fairly complicated equation with multiple parameters: ...
0
votes
0answers
9 views

Forecasting Incident ticket trend based on past data [on hold]

I'm trying to develop a prediction model the on Incident Ticket logged for my support queue for next 3 months, based on past 12 months data (trend). We need to consider the cause of creation of ...
3
votes
2answers
111 views

Difference between descriptive and predictive modelling

I was wondering if anyone could help me clear up the difference between descriptive and predictive modelling. I am trying to build a model to predict where house prices will go up. To do this I need ...
2
votes
2answers
79 views

What's wrong to fit periodic data with polynomials?

Suppose we have toy daily temperate data and we want to fit a model. A reasonable thing to do is fitting a periodic model with Fourier basis $$ f(x)=\beta_0+\beta_1 \cos(2\pi x/24)+\beta_2 \sin(2\...
1
vote
0answers
41 views

Importance interpretation with randomForest using R

I'm working on a predictive model using randomForest. I have some interesting results but the important variables got me confused. While using the varImPlot from ...
0
votes
0answers
9 views

How to fit a forecasting model on this irregular time series in Matlab to obtain predictions? [closed]

We have some data values from the past (1214415 to be exact over 1 minute intervals). We are only interested in the 'peaks', that is, how the trend pertaining to the maximum is increasing with time. ...
0
votes
2answers
26 views

How to calculate probability of success based on prior p-value

The FDA often requires a sponsor to conduct multiple clinical trials prior to approval. Given the following observations in a ph2 and ph3 trial, how would you go about predicting the probability of ...
2
votes
1answer
18 views

Are there any models that can handle out of sample features?

So I'm facing a regression problem where I have a categorical features (factor) where the levels are very commonly different between the training set and the test set. I have multiple measurements ...
0
votes
0answers
5 views

What are some viable models for all factor variables?

I have a dataset where everything, including my target dependent variable(s) are factors. Maybe I'm lacking creativity on this one, but what are some viable, predictive, models for something like this?...
1
vote
1answer
34 views

Should OLS always have a lower RMSE than Poisson Regression?

I'm working on building a predictive model for the number of singles a hitter in baseball generates over the course of a single game. Since the number of singles a hitter scores per game is count data ...
0
votes
0answers
7 views

what should be validation parameter for Logistic Regression(LR) in online learning plus rare event scenario?

We have been following below paper to predict CTR( Click probability) of different ad items. This will be used to serve different ads based on probability values. http://olivier.chapelle.cc/pub/...
0
votes
0answers
58 views

Prediction Algorithms in Statistics

Here it is what I'm looking for: Having a random variable, with an unknown distribution of its values, is there any smart algorithm that can predict the next value of the variable based on n samples ...
0
votes
1answer
19 views

Can I use information about the distribution of the dependent to improve prediction?

I'm trying to make predictions about a quantity on a per-subject basis. If I aggregate my complete sample I can get very good fit for distributions like gamma or Weibull, so I can make some ...
1
vote
0answers
24 views

Optimal prediction strategy with unstable parameters

I got a, at least for me, unusual data set at hand which I employ to create a prediction model for fresh data streaming in. For specific reasons, I only got "one shot" to build the model: That will ...
1
vote
1answer
20 views

Using different prediction models/algorithms for different subsets of dataset

Is there practice in data mining and machine learning where different parts of dataset are predicted using different algorithms/models? The logic is that some data samples are better predicted with ...
0
votes
1answer
37 views

Request for reference of cheat-sheet of paper concerning preprocessing and learning algorithms [closed]

Is there a comprehensive list of preprocessing steps that are highly recommended when using the classical learning algorithms (see below for a list of the families)? Is there a cheat-sheet or a cook-...
0
votes
0answers
10 views

Is it accepted to fit model with standardized data and predict on non-standardized data? [duplicate]

If you standardize your training data, then can it work on unstandardized data during predictions accurately? Many algorithms require the feature data to be standardized and I am wondering how/why/...
0
votes
0answers
18 views

Stock Price Prediction using Forward or Viterbi?

I've run a simulation with my data (using the Hidden Markov Model) and obtained my transition matrix, my hidden states and parameters. I am now wondering which algorithm would be able to predict (and ...
0
votes
2answers
26 views

Getting information from a GBM model

I built a gbm model (using caret package) in order to predict the probability of someone buy or not a car. However as this kind of model act as a black box my issue is to replicate the "profile" of ...
1
vote
4answers
93 views

How to increase the accuracy of my logistic regression model?

I am dealing with a tricky, unbalanced data set and trying to run a logistic regression model. One class is present with a 10:1 ratio. My objective here is to boost my predictive accuracy - minimize ...
0
votes
1answer
18 views

Predicting continuous position using input variables of unknown quality

The problem I'd like to solve can be reformulated as follows. Let's consider that I have to go to some parties and I would like to find out where in the room I am most likely to have a good time. I ...
1
vote
3answers
29 views

How should new variables to be added in logistic regression model in production?

We have built a LR model ( online SGD ) in Spark. There are more that 15 categorical variables only as independent variables.At run time new values come in few of the columns. We have used ...
0
votes
0answers
29 views

How to build model given data only?

Given data only and without any (prior) knowledge or information about the data, how to construct a model for the data and predict new data? There are many statistical models but I have no idea which ...
0
votes
0answers
30 views

What statistical analysis for identifying sections of

I am a stats novice, and dont quite know the method to use in my problem. So I have a set of independent variables, and I want to find what sort of prediction strength exists with some dependants. ...
0
votes
1answer
21 views

Chossing an algorithm when there is only one feature

I am combining multiple base classifiers for an ensemble classifier. Different sensors, such as an accelerometer, gyroscope and altimeter are classified individually, and their outputs are then fed ...
1
vote
1answer
35 views

Ensembles - few questions on approach with multiple models

I'm looking for some general-high level understanding of how I should apply ensemble techniques. I understand that some models can be already thought of as ensembles - such as random forests or some ...
1
vote
0answers
52 views

Time series prediction in R over more than 180,000 past data points takes forever?

We have data values pertaining to BPS (bits per second) traffic on a networking device. We have data from for a particular month (October) from the past 4 years. The data points are available in a 1 ...
0
votes
0answers
11 views

Projection Pursuit Regression

I am trying to use Projection Pursuit Regression to fit a model to my data set, but I am running into some difficulties. I have a few questions: 1. Can PPR only be used when you have many predictors? ...
0
votes
0answers
9 views

Predicting End Year based on Beginning Year - weighting time-based feature values

I apologize if this has been answered in the past, but I have not been able to find anything relative to this type of scenario. Our goal is to predict when an owner of an asset will sell that ...
0
votes
0answers
4 views

Is it sensible to roll up the revenue predictions?

I have a general doubt on how to make predictions. for example: Scenario A: Predict revenue by (Year | Quarter) Scenario B: Predict revenue by (Year | ProductA | ProductB | ProductC | ProductD) ...
1
vote
1answer
30 views

Comparing performance of two k-fold cross-validated models

I'm developing two k-fold cross-validated models, based on two different data sets, but using the same variables. I plan to then apply both models to each data set and calculate a few model ...
0
votes
0answers
53 views

Time series prediction in R where data is available over past 4 years in 1 minute intervals

We have data from 'october' in the past 4 years and we want to predict what data for this year is going to look like in October. The data looks like this: 1 2 3 4 5 6 ... Every october has 31 days, ...
0
votes
0answers
3 views

Reading time series data into R when you don't have time labels? [migrated]

We have a data set that is in a very strange format. It mentions the start date and the end date and time and then goes into data values. See the format sample below: ...
0
votes
0answers
12 views

Incorporating demographic features to the BTYD model

Currently I am working in a project which requires Buy Til You Die model.It takes into account only the purchasing history of a customer.I want to know how can I incorporate demographic feature of a ...
7
votes
1answer
383 views

When have I to stop looking for a model?

I'm looking for a model between stockprices of energy and the weather. I have the price of the MWatt bought between the countries of Europe, and a lot of values on the weather (Grib files). Each hours ...
0
votes
0answers
22 views

How can I overcome the “contrasts can be applied only to factors with 2 or more levels” when each factor has 1 level?

I have trained a model and now I want to predict the output for a new query with an svm. First, I got this error "length of 'center' must equal the number of columns of 'x'" which means that there ...
0
votes
0answers
16 views

Why regression trees in gradient boosting?

In the Gradient Boosting method, regression trees are used to model the residuals. Why specifically regression trees and not other? Can we use other?
1
vote
0answers
17 views

Pandas Time Series DataFrame Missing Values

I have a dataset of Total Sales from 2008-2015. I have an entry for each day, and so I have a created a pandas DataFrame with a ...
0
votes
0answers
24 views

Single pass object detection

Let we have a set of images $\{\mathcal I_i\}_{i=1}^n$ with labels $\{\mathcal B_i\}_{i=1}^n$, where each $\mathcal B_i$ is a set of regions. The problem is to find a function that given image $\...
3
votes
4answers
358 views

Logistic Regression: Does my model selection process make sense?

This is kind of a broad question and so I am okay with broad or general answers. In fact, each of these could be their own individual questions, but I think it makes sense to ask them all. Even if you ...