Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Prediction of unknown random quantities, using a statistical model.

0
votes
0answers
8 views

Mean squared error (MSE) prediction performance: Ridge vs Lasso?

It says that the ridge will outperform lasso in terms of prediction performance when the prediction metric is MSE, according to the answer to this post below: If only prediction is of interest, why ...
1
vote
0answers
12 views

Testing power of a predictive model on a very granular data

I recently got a review and resubmit from a journal, with a comment that the predictive power of my model is weak. The reason why one of the reviewers said it was the fact that my MAPE for a holdout ...
0
votes
0answers
16 views

Predict document topic with LDA using only document-topic probabilities

I did a LDA topic analysis to a corpus of N documents. Due to few unlucky events all word-topic -probability distribution matrices was lost and I am left with only the document-topic probabilities $P(...
0
votes
0answers
13 views

How to improve regression model and prediction results [closed]

How to improve regression results and find stronger predictors. This Data set is form UCLA. The results of the regression model on this data set are highly significant where my actual data has ...
0
votes
0answers
22 views

Predicting Sale Quantity

I am very interested in R and prediction models. I already used different models like ols, flexible ols, lasso, Regression trees and so on for american household data which was already perfectly ...
1
vote
1answer
57 views

Predicting probabilities after log-linear regression

I would like to estimate a log-linear regression and examine the results with Stata's marginsplot command. I have transformed my dependent variable into natural logarithm (to make a highly skewed ...
1
vote
2answers
30 views

Forecasting From an Age-based Distribution

I have an age-based probability distribution that looks something like this, where the age is in rows, and the year is in columns: ...
2
votes
0answers
25 views

Forecasting autoregressive model. What's the best linear predictor?

Obviously if $X_t = \phi X_{t-1} + Z_t$, then the best linear predictor of $X_t$ given $X_{t-1}$ is $X_t = \phi X_{t-1}$. But if $\phi$ is unknown, one may attempt to substitute $\phi$ by a Yule-...
0
votes
0answers
6 views

Least Angle Regression with Lasso: Results varying depending on seed for random numbers

p>>n binary classification with minority class. Split the data set into test and training using a seed. Applying LARS plus LASSO, yielded a nice 80% classification. However, if I vary the seed the ...
7
votes
2answers
1k views

Is it possible to combine predictions to improve overall prediction quality?

This is a binary classification problem. The metric that is being minimised is the log loss ( or cross entropy ). I also have an accuracy number, just for my information. It is a large, very ...
0
votes
1answer
36 views

Do Newey-West standard errors correct for Stambaugh bias?

I was wondering if Newey-West standard errors correct the Stambaugh bias when you have lagged stochastic regressors? The bias is also explained here. I know that Hodrick (1992) would correct for the ...
0
votes
0answers
25 views

Predicting a combination

Question Suppose we have a training set of families. Where each family is defined as such… Family: A list of integers. Each integer is the age of one of the family members. (e.g. with a 45 year ...
0
votes
0answers
11 views

Bug Prediction for next release

I have a requirement to predict the number of defect for next sprint release for each module. for example the data looks below ...
0
votes
0answers
20 views

How to obtain the estimated survival time of individual observations using survival analysis on churn dataset?

I have already done a survival analysis using cox regression on the churn dataset to find out the optimal parameters for getting the maximal survival time in my dataset. I have got a decent curve ...
0
votes
0answers
16 views

Modeling with different data types

My objective is to predict the taxi demand in Manhattan depending on time and location. In order to do so I divided Manhattan into census tracts, 288 spatial areas with approximately the same ...
0
votes
1answer
30 views

Are there models that can make prediction based on fixed value of a variable?

So I am making a time series analysis and even though packages like Prophet are good at it, I want to know how a certain value can change the prediction. For instance, I want to know how high the ...
0
votes
0answers
24 views

Counterfactual prediction with machine learning, sales data

I have a dataset from a supermarket with around 10 thousand products. The data has daily quantities and prices and discount information (whether the product had a discount and the size of the discount ...
0
votes
1answer
26 views

Error term average to zero?

I am quite new to statistic and regression. In one of literature, the setting for prediction to estimate f in regression, the error term averages to zero. What does it mean?
0
votes
0answers
9 views

Smoothing Predictions from HMM

I would say I do not have a strong foundation on stats, however, I am trying to use statistical tools for my research. I am using a hidden Markov model (HMM) to forecast day-ahead (hourly) solar ...
1
vote
0answers
21 views

Does this forecast visualisation make statistical sense?

The Met Office is the UK national weather service. Among it services, it provides assessment of upcoming storms in terms of their potential strength and consequences. This information is visually ...
0
votes
0answers
22 views

Predict effect of two features together from their respective, ordinal distributions

So, I have the results of two experiments, each of which examined a particular linguistic feature (feature 'A' or feature 'B'). Participants rated their impressions of these features across multiple ...
0
votes
0answers
9 views

what kind of prediction does predict.bess method use?

Seeking a way to use best subset selection from my coxph model, I found the package BeSS to be the only one out that that allows ...
0
votes
0answers
10 views

Bayesian prediction with minimum expected loss

I am studying for my machine learning exam and i have the following problem that i want to solve for preparation: I already solved Problem 1a, I have problems with 1b. After some google search I got ...
1
vote
0answers
9 views

How to test whether the mean of an observed time-series is statistically different from the mean of the predicted time-series?

Let's say I have an observed time-series from $y_1,...,y_T$, as well as an predicted time-series $\hat{y}_1,...,\hat{y}_T$. I want to test whether the mean of observed values $\bar{y}$ is equal to ...
0
votes
0answers
6 views

VGAM plotting predictions and confidence intervals genpoisson/generalized poisson

I fit a generalized poisson using VGAM and can output predictions using predict. However, the fitted.values are a matrix, ...
7
votes
3answers
837 views

Over fitting on purpose

Would it make sense to overfit a model on purpose? Say I have a use case where I know the data will not vary much respect to the training data. I'm thinking here about traffic prediction, where the ...
2
votes
0answers
37 views

Prediction close to a “pole”. What can I do?

I have x and y related in the way $x = a + \frac{1}{b+y}$. x is measured. I am interested in y. a and b are known. y must be $>=0$ from physics, x is also always $>0$. Solving for y: $y = \...
0
votes
0answers
8 views

Cluster 1D predictions based on surrounding categorial values

I am trying to predict soil layers. The image below shows the problem, left: the labels right: the predicted categories I want to be able to specify the beginning of the layer and the end of the ...
2
votes
1answer
23 views

Prediction with circular variable

Suppose I have a set of $N$ observations for a circular variable $\theta_i$. Each trial for which an observation of $\theta_i$ is taken can lead to an outcome $x_i \in \{0,1\}$. Next, suppose that the ...
0
votes
0answers
14 views

low prediction accuracy with unbalanced Dataset

I am working on a binary classification problem, where I have 2 classes (0 and 1). I have created a balanced Dataset with 70k samples(50% have the class 0 and the other 50 % the class 1), trained a ...
0
votes
1answer
25 views

Coordinates as model features

My goal is to predict the taxi demand depending on location and time in NYC. Hence, among other variables my dataset contains coordinates. My question is, can I use them as a predictor for my models? ...
0
votes
0answers
11 views

How does the tobit model relate to parametric survival model?

Is the Tobit model equivalent to a log-normal parametric survival model? If not, what are the pros and cons of a log-normal parametric survival model over a Tobit model?
0
votes
0answers
14 views

Best method for a prediction model with a spline term

I would like to run a prediction model on a dataset of about 500- outcome is binary. One of my continuous predictors is non-linear and I am interested in a total of 7 predictors. What is the best ...
9
votes
2answers
224 views

A ''significant variable'' that does not improve out-of-sample predictions - how to interpret?

I have a question that I think will be quite basic to a lot of users. Im using linear regression models to (i) investigate the relationship of several explanatory variables and my response variable ...
0
votes
1answer
16 views

How to deal with imbalanced data using logith algorithm

I have an imbalanced dataset for predicting bankrutptcy using the logit algorithm. My sample has 2%(200) bankrupt firms. Unfortunately my prediction is worthless with an auroc of 0.52. On top of that ...
0
votes
0answers
17 views

SVM prediction with historical data that starts at different times

I am working with an SVM prediction model that uses historical data starting in 2016. I now have new data that I want to use with the SVM model but it does not start until 2017. How can I use the two ...
0
votes
0answers
17 views

Prediction with categorical and continuous Variables

I want to predict the result of a match in a video game (win or loose). It's 5 players against 5 players game, who each plays a specific character. I have : the ID of each character (there are 150 ...
1
vote
1answer
31 views

How to Improve the relationship between predictors and observation for having a better fit

I am quite interested in the field of data analysis and mining and therefore, I have started doing some real problem using SVM regression to predict a target variable. My response variable is ...
0
votes
1answer
34 views

Is there a way to estimate the probability that the next number in a time series will be 0?

I have a set of 50,000 time series lasting 31 years (each time series for each of 50,000 cells). I would like to find the probability that the next year will go to 0 given the values in previous years....
1
vote
1answer
78 views

What more advanced methods can I use to predict future sales other than polynomial fitting?

I am using the polynomial fitting method to forecast the sales of a product throughout different years, where the polynomial is of degree 1. The error is measured by the sum of the squared residuals. ...
1
vote
0answers
25 views

How to work with an unknown dependent variable? [closed]

I am working on a used cars price prediction project using the Craigslist data. I have the car price provided by owners/dealers but it doesn't mean they can sell the car at that price. Is there a good ...
0
votes
0answers
14 views

Poisson data prediction with month variable

I have count data based on the number of failures per month. I have month, year, type of failure, and the count of fails. I have tried using R's glm package, ...
0
votes
0answers
8 views

Number of datas regarding the number of features

I'm trying to make predictions on a video game. The game is a 5v5 battle with champions. There are about 200 different champions. My input is a vector of 10 champion IDs (5 IDs for each team) and my ...
1
vote
0answers
47 views

Equation for standard error of linear predictor and 95% prediction intervals from logistic regression

I would like to present 95% prediction intervals in an online risk calculator. 1) After fitting a logistic model with lrm (which includes some restricted cubic ...
0
votes
0answers
9 views

Correlation between an ordinal categorical variable and a discrete numerical variable (in R)?

I am confused about which test to use for this particular problem. I have a data set which has questions received from clients and the complexity level the question was assigned by a team member. I ...
1
vote
1answer
26 views

In prediction, when should I use rolling windows vs. nonoverlapping ones?

Suppose I have daily time series data and I want to predict a month in advance using a set of features. I have lots of them so I'll be using regularized linear regression. To create the response I can ...
0
votes
2answers
24 views

MAPE results for the 4-week post-sample period

I'm trying to get the same results reported in the paper Taylor, J.W. (2003) Short-term electricity demand forecasting using double seasonal exponential smoothing. Journal of the Operational Research ...
1
vote
2answers
143 views

Holt-Winters function hw() in R

I would like to use the hw method from the R forecast package to predict electricity consumption. I tried to use it on the ...
0
votes
0answers
17 views

How many time series is it logical to model using VAR (in R)

I have weekly sales for around 6000 products, for which I would like to obtain forecasts for n periods. I believe that estimate ...
1
vote
1answer
27 views

What measurement of error should I use to compare predicted model results to actual measurements (and why)?

Lets say I have a time-series dataset of measurements g that varies with time t and I also have a time-series dataset of predictions of these measurements (lets call this g1) that again varies with ...