Predictive models are statistical models whose primary purpose is to predict other observations of a system optimally, as opposed to models whose purpose is to test a particular hypothesis or explain a phenomenon mechanistically. As such, predictive models place less emphasis on interpretability and ...

learn more… | top users | synonyms

0
votes
0answers
19 views

How to use a regression lm function used for prediction [migrated]

I apply a linear regression over a data set: ...
2
votes
1answer
31 views

Predictive model developement for logistic regression?

In the statistical courses I've taken, which are mostly introductory, when I have a model I would make hypotesis tests to reduce it to the simplest form and am effectively done. It is my understanding ...
-1
votes
1answer
20 views

Assessing the impacts of Illegal Bushmeat Hunting, Limpopo, South Africa [on hold]

If I want to assess and map the impact of illegal bushmeat hunting with camera traps. What would the best method be to statistically assess this data? Ideally, I would like to make a hotspots map to ...
1
vote
0answers
9 views

How to retrain a production classifier that blocks its own positive examples?

I'm looking for help understanding how to re-train a fraud detection classifier that's been deployed to production (where it successfully blocked much, but not all fraud coming into the system). I ...
0
votes
0answers
10 views

Access validity and reliability for hypotheses testing in prediction model

In my bachelor thesis I'm investigating a transaction data set of a retailer. I'm trying to find out indicators that predict a certain outcome, which is why I proposed several hypotheses for the ...
0
votes
1answer
30 views

model overfitting vs applicability

Consider two models I built: Model A I use a Neural Network to build a classification model and get a model that over fits , lets say the FPR in test set in 2 times that in train set. I am ...
0
votes
1answer
39 views

Which statistical models are suitable for prediction with clickstream data?

I'm a Statistics student, and I'm thinking of writing my master's thesis on clickstream data analysis. For my analysis I have a pretty big dataset (80 million rows), each of them being a click ...
0
votes
0answers
9 views

Can somebody describe what sparse models are in simple words?

I couldn't find any material that explains this. Is it also true that using Lasso regression will result in a sparse model only if there are any irrelevant variables/predictors?
0
votes
0answers
9 views

Replacing NA columns with Median in R [migrated]

I keep getting errors with the codes, which would be correct?
0
votes
0answers
9 views

An appropriate measure of predictive accuracy on non-uniform data?

When you have a model making a prediction about non-uniform data, how do you approach deciding on an appropriate measure of accuracy? For example, if the the data tends to be a normal distribution ...
3
votes
2answers
71 views

Do I do threshold selection for my logit model on the testing or training subset?

I have data with a binary outcome and I am doing logit model selection using AIC and BIC. I have already withheld 30% of the data as a holdout sample (testing subset) and used the remainder (training ...
0
votes
2answers
84 views

Model fitting: resampling the validation set to obtain distributions of test statistic

I see many descriptions of splitting the data set into a training part, a validation part and a test part. We train our models on the training part and choose the best model using the validation part, ...
3
votes
0answers
90 views

Which, if any, machine learning algorithms are accepted as being a good tradeoff between explainability and prediction? [on hold]

Machine learning texts describing algorithms such as gradient boosting machines or neural networks often comment that these models are good at prediction, but this comes at the price of a loss of ...
0
votes
0answers
15 views

Confusion over the test error and the expected error and actual implementation

Given a training set, $\tau = \{(x_1,y_1,\dots, x_N,y_N \}$ and a model $\hat{f}(x)$ has been fit. We have the following two definitions: The Generalisation (Test) Error $$ Err(\tau) = ...
2
votes
0answers
22 views

Difference between comparing forecasts and models

I started out looking for a way to test the difference between MSPE between two models (Question here), when (thanks to @Richard Hardy) I ended up reading a paper of Diebold regarding the ...
0
votes
0answers
6 views

One Response Value Fails for Proportional Odds Assumptions

I am analyzing placement scores on the response of students' grades for various courses. I have used a Chi-Sq test with the likelihood ratio to check the proportional odds assumption. For one of the ...
1
vote
2answers
57 views

Model prediction: test for difference in MSE

I have made two regression models. They were made on a training set of 80% of the data. And 20% of the data is the validation set. No test set is made. The models tell me how much premium a ...
0
votes
0answers
30 views

Better Predictive Model?

I'm running a bunch of different models trying to find one that is best at predicting using a Validation set and Root Average Squared Error(RASE) calculated from residuals as my main criteria. My data ...
-3
votes
0answers
30 views

Predictive modelling problem

I have a data set on Effort Deviation of software projects. The data is as follows: 985 data points. Response is continuous with negative, positive, and zero values. Input Parameters are 15 in ...
2
votes
0answers
14 views

dividing a multiply imputed dataset into derivation and validation cohorts

R/statistics noob. Mac OSX 10.11, RStudio 0.99.842. I'm developing a clinical prediction tool as part of my PhD. I have missing data (23k cases, 24 variables, 70% of variables have at least one ...
2
votes
0answers
18 views

Summing estimated probabilities in classification model

I am developing a series of predictive classification models (decision trees, neural networks, etc.) aimed at predicting the number of people who enroll in a program based on a variety of demographic ...
0
votes
1answer
18 views

Why do gam predictions not match gam smooth?

I am studying the effect of organic farming on honey reserves in honeybee colonies. I am trying to see how an increase in the percentage of organically farmed land (at various buffers around bee ...
0
votes
0answers
9 views

Difference between weighting and replicating observations in linear regression

I have a model in which each case is summary statistic of many observations. I am using a mixed effect model (lmer() in R) for prediction and I thought to give ...
0
votes
1answer
12 views

taking average of several models and feature sets

just a quick question that i cant seem to find a definitive answer for. When im doing feature selection, i end up with a list of the top performing sets. Would it make sense to use the top 10 sets ...
0
votes
0answers
47 views

Stuck while trying to predict on data based on H2O Deep Learning model

I have created an H2O Deep Learning model in R for multi-class classification and I want to use it to perform prediction. I would have assumed that if I use the model to predict on the validation ...
0
votes
0answers
11 views

Adversarial sequential learning with a linear model

I have a problem with the following characteristics: The value of an observation is a function of its predictors The nature of the relationship between value and predictors changes slowly over time ...
1
vote
0answers
19 views

Predicting occurrence of an event using time series data

I have data coming from sensors for 1 month. The data is time series with each data point separated by interval of 1 second. There are predictors like temperature, pressure, speed of the fan that has ...
0
votes
0answers
14 views

Method to determine whether or not users had a bad experience based on multiple variables: Average Bandwidth, Latency, and frame rate

I would like a recommendation on the best statistical method to use, as well as any suggested R packages to achieve this goal. I have three variables, Bandwidth, Latency, and frame rate for a set of ...
0
votes
0answers
7 views

A rate model for sodium channels

I am studying by myself Human Physiology. I have encountered the following question: In the following given model of sodium channel with 3 states open closed blocked (which I assume means ...
3
votes
1answer
28 views

Single prediction vs. summing more granular n-step ahead predictions

Say I want to predict the total rainfall for the next 365 days based on a set of predictors and daily historical observations. I could build a model that predicts annual rainfall and make a single ...
1
vote
1answer
24 views

With multinomial regression, how to predict an event and get the ROC curve?

I'm using the multinom package in R to run a multinomial logistic regression model. My dependent variable has 3 levels and as the output, I'm getting the ...
0
votes
0answers
12 views

predict a distribution of efforts over time

I am given data with efforts (the mythical man months) for many (ca 350) projects over 2 years. We want to use this data to predict the evolution of upcoming projects in terms of staff to find out ...
0
votes
0answers
11 views

adjusting nnet model for prediction

Could someone give some hints how to adjust paramters in nnet model for predictions ? I mean following parameters: maxit, range, decay, size, and ...
0
votes
0answers
12 views

How to get correlation of each predictor to response

I am wondering how can I get the correlation from one predictor to a repsonse when I am looking at a given data set with many predictors. For example, the output of GLM in R would be exactly what I ...
1
vote
0answers
11 views

Cross-classified multi-level model - application to marketing

I am working on predicting whether an individual customer will respond favourably to a marketing campaign (yes/no). I have data about customers, and their responses to previous campaigns. If possible, ...
0
votes
1answer
66 views

What model to use for predicting future expenses of an individual?

I am currently working on a Personal Finance application, which tracks expenses of a person. When entering an expense entry, the user selects the category of the transaction (e.g. 'Bills', 'Food', ...
0
votes
1answer
30 views

Can we use network neural for nonumeric data?

I am going to use network net package to do predidction. Tell me please: Is it neccessary to do scaling of data ? Is it neccessary to convert all data into numeric/int type ? I am newbie at this ...
0
votes
0answers
25 views

How to build a model from data with a proper hypothesis

I have a large dataset of items in a store and how they sell. It looks somewhat like this: ...
0
votes
0answers
9 views

Polynomial curve fitting for temperature prediction

First of all, I would like to say that I know very little about statistics. I need to make a C# application to predict three days weather for school project and need some model and have been exploring ...
0
votes
1answer
6 views

Classification of temporal instances

Problem description I am working for a telecom domain project where I am tasked to predict whether a customer would dispute his/her monthly bill or not. I have following historical data elements ...
5
votes
1answer
80 views

Reproducible benchmarks for the performance of statistical prediction methods?

There are many statistical models used for predictive modelling. These include famous methods such as naive Bayes, knn, SVM, random forest etc. I am looking for reproducible examples (preferably in ...
1
vote
1answer
34 views

Machine learning step order question

I have been working on this project for over a year now and I believe i finally have things figured out. Mainly i'm looking for any suggestions or things i'm doing wrong with my process, but i also ...
0
votes
0answers
18 views

multi linear regression model. Modeling with a heavily skewed binary independent variable

Dataset and goal: One continuous measurement( to be modeled as a dependent variable) and four other measurements (one binary and the rest are category variables with multiple levels) to be modeled as ...
0
votes
1answer
60 views

Noob Data Design Question - 1 on 1 competition

My question is about how to best structure my dataset for a competition between 2 players of a game (for the purposes of prediction of future game winners). There will potentially be hundreds of ...
0
votes
1answer
11 views

Conditional probability sub-model so solve setting with a factor that has many levels

I stumbled upon a post of the http://www.win-vector.com/ blog where they treat the problem when a factor with many levels occurs. In my understanding instead of using the factor itself, they use the ...
0
votes
0answers
12 views

Topics needed to learn to for predicting future of a data set?

I am planning to work this summer by writing a code which will eventually become an app, that has the ability to predict what the prices will be atleast 6 month ahead of time. I have a data set of the ...
1
vote
0answers
30 views

Organic vs Paid Attribution Model (Granger)

I'm wondering if there is literature or studies done on how to model organic attribution from paid user acquisition. So the context is, on our mobile app, we have paid installs that we purchase and ...
0
votes
1answer
42 views

ARMA lag selection for ARMA-GARCH models

When I read this group questions about lag selection for ARMA part of ARMA-GARCH models I found 2 different answers from moderator: The use of GARCH and ARMA GARCH estimation process in practice I ...
0
votes
0answers
19 views

Time Series Modelling With Two (or more) Periodic Components

I'm trying to create a model to predict hourly electricity usage. Looking at the data, it appears that there are three different components that I'm going to want to capture in my model. First, there ...
2
votes
0answers
23 views

How to model timeseries temperature data?

I would like to model a timeseries consisting of internal temperature data of a greenhouse, collected at 15 min interval and then use it to make predictions in the future. This is how my data looks ...