Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [prediction]

Prediction of unknown random quantities, using a statistical model.

0
votes
0answers
21 views

predict continuous variable over time for each row

I have many items with a start date, which produce continuous outputs over time for a few months after the start date. These outputs often follow a decay function but not always. So it is a bit like a ...
0
votes
1answer
27 views

Is it overfitting if I am using predictions from cross-validation as a level 2 feature for stacking model?

I am learning how to stack models, but I am worried if this is not a practical way to do it. I am using the full dataset and using cross_val_predict to get the ...
0
votes
0answers
26 views

How to deal with a negative Kappa in classification?

I have a dataset with one binary class to be predicted, with 18 binary predictors and 17400 rows. Here I used a stratified split, with approximately 85% (14648 rows) for training and 15% (2752) for ...
0
votes
0answers
11 views

How do I extrapolate website click data for subsequent months?

Started my first job and prolly my first post here too - any help would be greatly appreciated :D Please find the question in the attached image.
0
votes
0answers
18 views

Maximize mtry in random forest to hone in on most important predictors?

I'm currently using random forest to determine how a set of temporal predictors, lagged and aggregated at many different timescales, influences a binary disease outcome. My main goal is to understand ...
0
votes
0answers
22 views

Reporting mean-squared error using test set

I'm reading an article in which the mean-squared error of the model $$y=X\beta + \varepsilon,~~E(\varepsilon)=0\text{ and }Var(\varepsilon) = \sigma^2$$ is defined as $$ME = E[X\hat{\beta} - X\beta]^...
1
vote
1answer
18 views

How to model a binary vector by the Beta-Bernoulli distribution in R or Python?

I have a set of data $x=(x_1,x_2,x_3,...)$;e.g:$x=(0.1,0.003,0.78,...)$ which follows a beta distribution. Since $x_i$ in my data $x :x_i ~ Bernoulli(p)$, where $p$ is the prior of average $x$ and ...
1
vote
0answers
11 views

uncertainty in dependent variable at single covariate

I have a dataset consisting of responses of a dependent variable measured at the same independent variables over multiple trials. It looks something like trial $i$: $(x^{(i)}_1, y^{(i)}_1) = (1.0,\...
0
votes
0answers
9 views

Predict return value and evaluate with information ratio

I want to predict values on return-T2 having this data available: 'DATE', 'RETURN-T1', 'RETURN-T2', 'OTHER_VARIABLE', 'OTHER_VARIABLE',... I thought of doing feature selection and training a model ...
2
votes
1answer
196 views

Auto-Arima creates a straight line help

I'm trying to create a forecast using autoarima with some data, but i always get a straight-line, can someone please help me? :) This is what i've got so far ...
0
votes
1answer
21 views

Setting the correct cutoff for binomial GLM's predicted probabilities

I'm using a binomial GLM to model what kind of student would pass or fail a certain class. When I use predict with type "response" on my model, I see a vector of probabilities. Per my understanding, ...
0
votes
0answers
13 views

Selection of regression model for prediction and interpreting quadratic regression results

I am regressing between the body mass and eye diameter in different bat species. The relationship is non-linear (picture attached) as the eye-size cannot increase linearly with respect to body size ...
0
votes
1answer
13 views

How can I implement univariate Nadaraya-Watson regression for prediction?

How can I implement univariate Nadaraya-Watson regression for prediction? And what is the $x ,x_i$, and $y_i$? How can I select the $x ,x_i$, and $y_i$? The sample for prediction and the shape of ...
0
votes
1answer
22 views

Predicting survival time from log-hazard

I have estimated a Weibull regression model in BUGS/JAGS which gives me the log-hazard as a function of intercept (baseline hazard) and covariate effects. The intercept is estimated as -9.826 and one ...
2
votes
1answer
20 views

Why does adding a redundant predictor to randomForest improve prediction?

I write in hopes of understanding an odd behavior of the randomForest package. I am trying to predict a factor y with 9 levels using 8 binary factors X1-X8. I get good accuracy (0.8959), and the ...
0
votes
0answers
23 views

Alternatives to logistic regression when dataset can't meet 10 events per predictor variable

I have a dataset with a binary outcome I'd like to model, but have many more parameters than events, probably ~100 categorical parameters to test, in a dataset with 20 events in ~600 observations. As ...
0
votes
0answers
2 views

How to predict unknown(hidden) value by incomplete value or partly recorded value

Let me make it clear by make an example: Suppose I knew a person's cost each month for 3 years like: 2016Jan : $2500 2016Feb : $4000 2016Mar : $3500 ... Just according to this, can I predict how ...
2
votes
0answers
25 views

Residual-analysis by running KNN or Clustering on incorrect ANN or RF predictions? [on hold]

I've outputs of some neutral net models, which naturally gives some errors in the prediction. What I am interested in is determining whether there are any systemic patterns associated with these ...
0
votes
0answers
9 views

Measuring predicitve accurateness relative to mean squared error

Imagine that $N$ agents try to predict t $k$ values $e_1, ..., e_K$, that differ in their 'predictability', i.e. some values are much easier to predict than other values. I am trying to define a ...
0
votes
2answers
31 views

Demand prediction multiple target variables

I am working on prediction of porduct demand. I have dataset of transactions for many customers for a whole 2017 year. Problem is that I have lot of porducts with different quantities and packages in ...
0
votes
1answer
34 views

Frequentist definition of probability and prediction?

The frequentist definition of probability states that: The probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible when nothing leads us to ...
0
votes
0answers
15 views

Scoring System via Elastic Net and Bootstrapping a reasonable approach?

I am still at model Building stage. It's for heart surgery data, Building a scoring System for (near-future) death by a set of variables (~20) on a cohort of about 500 patients, 2 competing therapies, ...
0
votes
0answers
31 views

Penalty matrix on Decision Tree

I am doing a project on predicting credit risk and am trying to use decision trees. First, I trained the tree without any penalty matrix. Next, I defined my penalty matrix as $$ \begin{matrix} 0 &...
1
vote
1answer
44 views

Reverse prediction in a time series

We know using models like ARIMA we can do out of sample predictions for a Time Series. i.e. we can know what would be the value v at time t. Can we do the reverse of it, and find at what t will be v a ...
3
votes
0answers
36 views

Noise in ARIMA Model In-Sample Predictions

I am working on fitting some financial data into an ARIMA model to give me a forecast of the next time period. I am using pyramid's auto_arima function to get a ...
2
votes
0answers
60 views

Optimism bootstrap with non-linear models

I have come across an example in my research with heavily overfit non-linear probabilistic classifiers, where the optimism bootstrap appears to underestimate the optimism, even when using a proper ...
3
votes
4answers
71 views

Should we include random error term when doing prediction using regression models

I have been confused by this problem for a long time. That is when we are making prediction using regression models, the regression function only tells us the predicted mean value of the dependent ...
4
votes
1answer
48 views

How to test whether 2 prediction intervals are statistically different?

I've been struggling with this for a while now, hopefully someone will know how to help me :) Here it is : 1) I'm using a linear mixed effects model on longitudinal data (biological values of many ...
0
votes
0answers
16 views

Predict future result when another variable gets a fixed value

I have the weekly cost and revenue data for 50 weeks Date,Cost,Revenue ... ... 1/10/2018,10,8 8/10/2018,13,10 15/10/2018,13,11 I want to predict the Revenue for ...
1
vote
1answer
70 views

Mean squared error (MSE) prediction performance: Ridge vs Lasso?

It says that the ridge will outperform lasso in terms of prediction performance when the prediction metric is MSE, according to the answer to this post below: If only prediction is of interest, why ...
1
vote
0answers
22 views

Testing power of a predictive model on a very granular data

I recently got a review and resubmit from a journal, with a comment that the predictive power of my model is weak. The reason why one of the reviewers said it was the fact that my MAPE for a holdout ...
0
votes
0answers
59 views

Predict document topic with LDA using only document-topic probabilities

I did a LDA topic analysis to a corpus of N documents. Due to few unlucky events all word-topic -probability distribution matrices was lost and I am left with only the document-topic probabilities $P(...
0
votes
0answers
26 views

Predicting Sale Quantity

I am very interested in R and prediction models. I already used different models like ols, flexible ols, lasso, Regression trees and so on for american household data which was already perfectly ...
1
vote
1answer
118 views

Predicting probabilities after log-linear regression

I would like to estimate a log-linear regression and examine the results with Stata's marginsplot command. I have transformed my dependent variable into natural logarithm (to make a highly skewed ...
1
vote
2answers
31 views

Forecasting From an Age-based Distribution

I have an age-based probability distribution that looks something like this, where the age is in rows, and the year is in columns: ...
2
votes
0answers
30 views

Forecasting autoregressive model. What's the best linear predictor?

Obviously if $X_t = \phi X_{t-1} + Z_t$, then the best linear predictor of $X_t$ given $X_{t-1}$ is $X_t = \phi X_{t-1}$. But if $\phi$ is unknown, one may attempt to substitute $\phi$ by a Yule-...
0
votes
0answers
7 views

Least Angle Regression with Lasso: Results varying depending on seed for random numbers

p>>n binary classification with minority class. Split the data set into test and training using a seed. Applying LARS plus LASSO, yielded a nice 80% classification. However, if I vary the seed the ...
7
votes
2answers
1k views

Is it possible to combine predictions to improve overall prediction quality?

This is a binary classification problem. The metric that is being minimised is the log loss ( or cross entropy ). I also have an accuracy number, just for my information. It is a large, very ...
0
votes
1answer
44 views

Do Newey-West standard errors correct for Stambaugh bias?

I was wondering if Newey-West standard errors correct the Stambaugh bias when you have lagged stochastic regressors? The bias is also explained here. I know that Hodrick (1992) would correct for the ...
0
votes
0answers
26 views

Predicting a combination

Question Suppose we have a training set of families. Where each family is defined as such… Family: A list of integers. Each integer is the age of one of the family members. (e.g. with a 45 year ...
0
votes
0answers
25 views

How to obtain the estimated survival time of individual observations using survival analysis on churn dataset?

I have already done a survival analysis using cox regression on the churn dataset to find out the optimal parameters for getting the maximal survival time in my dataset. I have got a decent curve ...
0
votes
0answers
17 views

Modeling with different data types

My objective is to predict the taxi demand in Manhattan depending on time and location. In order to do so I divided Manhattan into census tracts, 288 spatial areas with approximately the same ...
0
votes
1answer
32 views

Are there models that can make prediction based on fixed value of a variable?

So I am making a time series analysis and even though packages like Prophet are good at it, I want to know how a certain value can change the prediction. For instance, I want to know how high the ...
0
votes
0answers
30 views

Counterfactual prediction with machine learning, sales data

I have a dataset from a supermarket with around 10 thousand products. The data has daily quantities and prices and discount information (whether the product had a discount and the size of the discount ...
0
votes
1answer
27 views

Error term average to zero?

I am quite new to statistic and regression. In one of literature, the setting for prediction to estimate f in regression, the error term averages to zero. What does it mean?
0
votes
0answers
15 views

Smoothing Predictions from HMM

I would say I do not have a strong foundation on stats, however, I am trying to use statistical tools for my research. I am using a hidden Markov model (HMM) to forecast day-ahead (hourly) solar ...
1
vote
0answers
23 views

Does this forecast visualisation make statistical sense?

The Met Office is the UK national weather service. Among it services, it provides assessment of upcoming storms in terms of their potential strength and consequences. This information is visually ...
0
votes
0answers
22 views

Predict effect of two features together from their respective, ordinal distributions

So, I have the results of two experiments, each of which examined a particular linguistic feature (feature 'A' or feature 'B'). Participants rated their impressions of these features across multiple ...
0
votes
0answers
11 views

what kind of prediction does predict.bess method use?

Seeking a way to use best subset selection from my coxph model, I found the package BeSS to be the only one out that that allows ...
0
votes
0answers
13 views

Bayesian prediction with minimum expected loss

I am studying for my machine learning exam and i have the following problem that i want to solve for preparation: I already solved Problem 1a, I have problems with 1b. After some google search I got ...