Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Predictive models are statistical models whose primary purpose is to predict other observations of a system optimally, as opposed to models whose purpose is to test a particular hypothesis or explain a phenomenon mechanistically. As such, predictive models place less emphasis on interpretability and ...

2
votes
1answer
18 views

How can a person predicted best playing 11 in a match between two teams?

https://www.dream11.com/games/cricket/point-system This website allows people to bet on cricket and football matches. They ask people to select 11 players and there are point system, so at the end ...
1
vote
0answers
9 views

uncertainty in dependent variable at single covariate

I have a dataset consisting of responses of a dependent variable measured at the same independent variables over multiple trials. It looks something like trial $i$: $(x^{(i)}_1, y^{(i)}_1) = (1.0,\...
0
votes
0answers
3 views

What is minCases in C5.0Control using R

from Package (C5.0 Decision tree Using R ) definition "minCases : an integer for the smallest number of samples that must be put in at least two of the splits." I very confuse about it . Please ...
15
votes
1answer
425 views

Stepwise AIC - Does there exist controversy surrounding this topic?

I've read countless posts on this site that are incredibly against the use of stepwise selection of variables using any sort of criterion whether it be p-values based, AIC, BIC, etc. I understand why ...
3
votes
2answers
52 views

Machine Learning - Prediction Interval - Cheating?

I work at a company that is trying to use machine learning methods in particular gradient boosting and neural networks to make predictions on stock market data, so using historical data to predict ...
0
votes
0answers
9 views

Forecasting/predicting techniques for qualitative data?

I have a food alert dataset composed of nominal qualitative variables, such as type of alert, country of origin, action taken, etc. as well as the date on which the alert was recorded. What ...
0
votes
0answers
2 views

How to predict unknown(hidden) value by incomplete value or partly recorded value

Let me make it clear by make an example: Suppose I knew a person's cost each month for 3 years like: 2016Jan : $2500 2016Feb : $4000 2016Mar : $3500 ... Just according to this, can I predict how ...
0
votes
1answer
15 views

Polluting a dataset with “non-determinables”

Let's say I'm working on solving sudoku puzzles with machine learning. Now, plenty of good methods exist for solving sudoku algorithmically, no machine learning required, but let's play along to get ...
0
votes
0answers
11 views

Repeated CV evaluation with confidence intervals in R caret?

it occurs to me that there is a part of model evaluation that I have not understood yet. The problem that I am working on now illustrates the point well I think. I need to fit a model of >400 ...
1
vote
1answer
24 views

Posterior and Predictive Density [closed]

Let X1 be a claim from an auto insurance policy. Suppose X follows an exponential distribution with rate lambda, where lambda follows a gamma distribution with mean 2 and variance 2. What is the ...
0
votes
0answers
37 views

multi-response forecasting

General Dear community, I really struggle with some imporant issues for my next project. In general, the investigation is about multi-response forecasting with financial data. The predicability of ...
2
votes
1answer
38 views

Should PCA be (always) done before Naive Bayes classification

According to Wikipedia page on Naive Bayes: .. Naive Bayes classifiers are a family of simple "probabilistic classifiers" based on applying Bayes' theorem with strong (naive) independence ...
0
votes
0answers
15 views

Predicting user active time in a website

I am very new in data science and machine learning. I need to find out/predict a time in which the user is active in a website during the day. I have a dataset with 3 columns listed as "user_id", "...
0
votes
1answer
25 views

developing and assessing a prediction Cox model using lasso

I wonder if anyone can comment on if the following modelling strategy is valid please? I have a 200 patient survival data set (actually 2 data sets: 40 events and 160 events) and 100,000 ish candidate ...
0
votes
0answers
12 views

find patterns in binary data set?

I have a data set of movement (rotation around 360 degree axis)of an object .I just need to find a pattern with in the data so that I can predict out which all regions get triggered when the object ...
0
votes
0answers
20 views

Repeated Measures to Create Bigger Dataset for ML?

Background: I work for a small health center, and we're interested in predicting adverse events like hospitalizations. We are not interested in generalizable knowledge (effect sizes of predictors, ...
3
votes
1answer
45 views

How to handle low correlation regression problem in building the predictive model

This question is related to a rainfall prediction problem where I am trying to predict the amount of rainfall based on the meteorological features such as Max Temperature, Min Temperature, Dry Temp, ...
0
votes
0answers
14 views

Is there any algorithm combining classification and regression in order to reduce the error due to misclassification?

I am aware about this post but I have a different question. I have a dataset with two classes. I can train a classifier and a regressor to the dataset per class. Then I can predict the class for a ...
2
votes
2answers
33 views

How would you go about modelling a sport with a fixed maximum score line (e.g. volleyball or tennis?)

In short, I don't have a great maths background but have always mucked about with modelling sports and making predictions (as I enjoy it). I have always used the Hal Stern "The Probability of Winning ...
3
votes
0answers
40 views

Relationship between Total Over/Under scores and actual total scores in sports

I have a data set of actual scores from sporting games, matched with the bookmaker's Total Over/Under Score (O/U Score) and the odds the bookmaker was offering that the game's total score would fall ...
1
vote
0answers
13 views

What does it mean when mse and result from regression methods are almost the same?

I try to use linear, ridge, lasso regression to predict, and I give them 2 features. There are 100 samples in train data and 30 samples in test data. Something confused me is the predicted value and ...
1
vote
1answer
40 views

Linear Model - accuracy of predictions with confidence interval

I am fitting a linear model to predict a variable which is a type of performance of animal behaviour. Let's call it performance. When the model makes a prediction ...
6
votes
1answer
85 views

Why are natural splines almost always cubic?

By natural spline, I mean a regression spline that's linear at the boundary (i.e. the regions where X is smaller than the smallest knot or larger than the largest knot). I know that for smoothing ...
1
vote
0answers
28 views

Estimation Multinomial Logit [closed]

I need to create a code manually corresponding to the likelihood of the multinomial logit model in R. I have not been able to get the same results from some packages (mlogit, multinom). My database ...
0
votes
0answers
14 views

Model selection and conclusion on prediction

I am predicting the taxi demand in NYC depending on time and location. So far, I am considering 3 models: linear regression ridge regression random Forests In order to improve linear regression I am ...
35
votes
5answers
4k views

How to handle a “self defeating” prediction model?

I was watching a presentation by an ML specialist from a major retailer, where they had developed a model to predict out of stock events. Let's assume for a moment that over time, their model ...
1
vote
1answer
26 views

Sales Forecasting for Multiple Dealers

I have a data-set containing 7046 unique dealer codes and their monthly sales data from April 2013-August 2018. The Financial Year for the sales data begins in the month of April for a year and ends ...
0
votes
0answers
19 views

Using model from one data set to predict results for another data set

I'm not certain how to phrase this question: I have a dataset of ~45000 execution times of two sets of data. Approximately 35000 of these execution times is ran in one environment, and the remaining ~...
0
votes
0answers
19 views

How can we predict reliability for type 1 censored data?

I am working on a somewhat challenging problem. I have data for a large set of equipment. My main research question is: How many units will fail in each future month in the coming decade? So far I ...
0
votes
0answers
20 views

ML Regression: Rounding up -ve values for predictions to 0 (if -ve outcomes are impossible) before testing algorithm accuracy?

I am working on a Machine Learning linear regression problem where the output cannot be negative. However when I am running my learning algorithm, the predictions for low values in the test data tend ...
0
votes
0answers
10 views

Testing model trained on Standardized/Scaled data

I have a dataset that was passed to a StandardScaler before being passed into a classification model for training. StandardScaler was also applied to the test data for Model validation. But, the ...
0
votes
1answer
47 views

how to choose model when training accuracy is lower than validation accuracy while training neural network?

Below is a specific case but a general situation i find myself involved with in my job. This question is intended at getting ideas on how to pick the best model: Dataset: rows: 10,166, features: ...
0
votes
0answers
15 views

SVM Regression n R

Hi The following graph represents original data, linear regression and Support vector regression. I would like to know if this is a decent plot of SVR and not understanding how to predict for a new ...
0
votes
0answers
85 views

Deriving predictive distribution

In Bayesian Regression, I am confused how to to get $f*$ and $\sigma*$, given $$y^∗ \mid \vec{y}\sim\mathcal{N}(f^∗ , σ^∗ )$$ $$ p(y^* \mid \vec{y}) = \int{p(y^* \mid \vec{w}) p(\vec{w} \mid \vec{y})...
0
votes
0answers
24 views

Non-orthogonal experimental design and model selection

I am working on designing some chemical experiments, with the goal (for now) to optimize reaction yield. I intend to use principal component scores in order to investigate solvents, Lewis acids etc. ...
0
votes
0answers
21 views

Statistical model choice for historical elections?

I'm interested in exploring the effects of changes in voter turnout on historical elections. (For example, how would voter turnout have had to change for candidate X to win a senate seat in 2008 ...
0
votes
0answers
16 views

What should be the right model to analyse this data? Should I use time series analysis?

I got a biomanufacturing dataset and try to build a model for the crystallization part. There are two variables: the temperature (T) of the tank (which is changing over time but specified by the ...
0
votes
0answers
20 views

Compare predictive discrimination between joint model and Cox model using a AUC measure

I have two models; 1) the joint model including baseline predictors for mortality and the longitudinal biomarker as an additional predictor, and 2) a Cox model including only the baseline predictors ...
1
vote
1answer
47 views

Using a logistic regression to predict binary outcome

Any input on my following query would be great. Currently I have a a number of monthly datasets of individual(s) who have purchased properties (2012-2017). These individuals can be split into two ...
0
votes
0answers
19 views

What is null distribution of brier's score or logarithmic score? How to test them properly?

I would like to get some p-values for predictions in an independent test set according to proper scoring rules. So I would like to know if my brier score/logarithmic score is statistically ...
0
votes
1answer
42 views

Do Newey-West standard errors correct for Stambaugh bias?

I was wondering if Newey-West standard errors correct the Stambaugh bias when you have lagged stochastic regressors? The bias is also explained here. I know that Hodrick (1992) would correct for the ...
0
votes
0answers
7 views

Performance of Hierarchical Temporal Memory on unsupervised online anomaly detection problems

I'm looking for an algorithm to detect anomalies in streaming data (server metrics). The detection needs to be near-real time and unsupervised (labeled data will never be available, unfortunately, and ...
0
votes
2answers
34 views

Which data model to use for nominal independent variables and continuous dependent variable?

I have a data set with two nominal features (which are my independent variables) and a continuous numeric output variable (i.e. dependent variable) between range of -10 to 10. What kind of predictive ...
3
votes
0answers
22 views

Correct cross-validation procedure for single model applied to panel data

Questions What is the correct CV procedure for panel data? I've been thinking of the problem as cross-validating a model fit to multiple time series data. Is the "population informed" CV procedure ...
1
vote
0answers
7 views

Is it possible to “tune” a trained model in one population so it can be used for a different population (i.e., by swapping variables)?

Say, I have trained a model to classify patients into cardiovascular disease (CVD) and non-CVD. The model building process is as follows: There is a gold standard to compare the model with. The ...
0
votes
0answers
4 views

Predict setting parameters for machine based on 2 quality values

I'm getting started with Machine Learning. I'm not sure which approach would be the best for the following scenario: I want to build a model that predict optimiced setting values for a machine, based ...
0
votes
0answers
18 views

Predicting zeros with hurdle package

I am trying to predict using the countreg::hurdle() in R. I've realized that hurdle is working well with models with a lot of 0s. However, I'm confused by its ...
6
votes
2answers
130 views

Using partial measurements of output variable in modeling

My question is: How can we use partially measured output data in a training set? This is vague, so I concretize it in a whimsical tale. Squirrels Have Nuts, But How Many? Setup There is a set $S$ ...
2
votes
1answer
103 views

Example when using accuracy as an outcome measure will lead to a wrong conclusion

I am looking into various performance measures for predictive models. A lot was written about problems of using accuracy, instead of something more continuous to evaluate model performance. Frank ...
0
votes
0answers
31 views

Predicting future spend with little direct historical data

I have a dataset of credit card customers. The dataset consists of past cohorts of supposedly similar card customers and I'm meant to generate forecasts for customer spend for a recent cohort (or even ...