Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Prediction of unknown random quantities, using a statistical model.

1
vote
0answers
12 views

Equation for standard error of linear predictor and 95% prediction intervals from logistic regression

I would like to present 95% prediction intervals in an online risk calculator. 1) After fitting a logistic model with lrm (which includes some restricted cubic ...
0
votes
0answers
3 views

Model deployment - the right data batch structure

This question goes mainly out to the data scientist and data engineers that have applied experience, although anyones 2cents are appreciated. Intro: I have a production ready model that has been ...
0
votes
0answers
6 views

Correlation between an ordinal categorical variable and a discrete numerical variable (in R)?

I am confused about which test to use for this particular problem. I have a data set which has questions received from clients and the complexity level the question was assigned by a team member. I ...
1
vote
1answer
22 views

In prediction, when should I use rolling windows vs. nonoverlapping ones?

Suppose I have daily time series data and I want to predict a month in advance using a set of features. I have lots of them so I'll be using regularized linear regression. To create the response I can ...
0
votes
2answers
18 views

MAPE results for the 4-week post-sample period

I'm trying to get the same results reported in the paper Taylor, J.W. (2003) Short-term electricity demand forecasting using double seasonal exponential smoothing. Journal of the Operational Research ...
0
votes
0answers
19 views

R neuralnet package is very slow for classification project [closed]

I am running neuralnet function to create classification model on a data set, which has 21 input variables, one response variable with two possible classes and around 3400 values. I transformed it to ...
1
vote
2answers
34 views

Holt-Winters function hw() in R

I would like to use the hw method from the R forecast package to predict electricity consumption. I tried to use it on the ...
0
votes
0answers
14 views

mlogit in R - tips on improving accuracy [closed]

I'm working in R, and trying to see how I can improve on my model. As of now my mlogit looks something like M <- mlogit(Choice ~ alternative | independent) and I'...
0
votes
0answers
17 views

How many time series is it logical to model using VAR (in R)

I have weekly sales for around 6000 products, for which I would like to obtain forecasts for n periods. I believe that estimate ...
1
vote
1answer
22 views

What measurement of error should I use to compare predicted model results to actual measurements (and why)?

Lets say I have a time-series dataset of measurements g that varies with time t and I also have a time-series dataset of predictions of these measurements (lets call this g1) that again varies with ...
-1
votes
0answers
10 views

Machine Learning methods for Multiple Response Prediction [closed]

What are the different machine learning techniques available for predicting multiple dependent variables? Can we use deep learning neural networks?
1
vote
1answer
41 views

How to check if a polynomial regression has any predictive value? [closed]

After fitting the polynomial data to a given curve, how can I check which of the many curves has the most predictive value ?
1
vote
1answer
31 views

Multicollinearity and predictive performance

Looking at this statement: "Multicollinearity does not affect the predictive power but individual predictor variable’s impact on the response variable could be calculated wrongly." Is this ...
2
votes
1answer
30 views

Bayesian predictions

I have a more general query about some confusion ive been having lately deciding what the bayesian predictions are in my model. Suppose I have a model, $y_{j} = \mu_{j} + e_{j}$, (for which i have ...
0
votes
0answers
21 views

Random effects in gamlss

I have a question regarding the gamlss package. I am attempting to fit a mixed effects model using the Befa Inflated distribution as follows ...
1
vote
0answers
27 views

Specific Predictions (Binary) in R

I want to make two new predictions (based on two new data frames). What I did: First of all (obvious) splitting the data: train / test Secondly: train my model (and evaluate it on my test set) ...
3
votes
1answer
29 views

Which model assumptions are important for prediction?

Disguised as other questions, there are frequent questions where the OP checks for violations of model assumptions (e.g. normality, homogeneity of variance in linear regression) in models they intend ...
6
votes
2answers
175 views

Is it wise to use predicted values to model predicted values further down the line?

Hi I have two questions that are related. I am wanting to model sales for different areas in a business and have been looking at ARIMA, I am not too happy with the results of this especially when I ...
0
votes
0answers
25 views

Categorising US prescription data as 'Loading Dose' or 'Maintenance Dose'

I have a dataset of patient prescriptions that I need to label 'Loading Dose' [LD] or 'Maintenance Dose' [MD]. Only the MD part of treatment is thought effective and I need to run some models on this ...
-1
votes
1answer
25 views

What neural network topology is recommended to predict signals?

Assumed I have gathered some measures of a complex system and I want to train a neural network to predict certain measure outcomes based on some simulated inputs, what ANN layout would you recommend? ...
2
votes
0answers
22 views

AutoML / Data science - Prediction websites [closed]

Are there any websites where we can simply upload a training dataset with labels, and get predictions for a test dataset ? Google AutoML and MIT ATM would be similar to what I'm looking for, but they ...
1
vote
0answers
20 views

Poisson processes or regression for lambda

I am asked to solve a problem where I have a machine producing toys in tho slots and want to predict number of faulty toys. The data is like this: ...
4
votes
1answer
106 views

Proof that predictions are unbiased in in endogenous linear model

Problem Statement Suppose we have a linear model given by $$y = X\beta + \varepsilon,$$ where $\varepsilon\sim N(0, \sigma^2 I)$ and $E[\varepsilon|X]\neq0$ (i.e., explanatory variables are ...
1
vote
1answer
22 views

R: Predicting future panel data

I have an unbalanced panel covering the years 2000-2017 for multiple countries (some missings as unbalanced suggests), for which I ran a fixed effects model with plm...
1
vote
1answer
44 views

Ridge\Lasso — Standardization of dummy indicators

Say I have a data set with say 5000 rows and about 150 columns (5000 samples, 150 predictors/features) and I'm interested in a applying a ridge or lasso regression. (Let us assume using a logit link ...
0
votes
2answers
59 views
0
votes
1answer
22 views

Establish the proper prediction threshold

Suppose for a ML classification problem X, I have three labels, i.e. 1, 0 and -1. For some reasons, I obtained bad accuracy. So I decided to change the number of labels to two, i.e. 1 and -1. That way ...
0
votes
0answers
39 views

test joint equality of different regression models' predictions at a given point

We have $k$ datasets, each consists of $Y^{(i)}$ and $X^{(i)}$ for $i = 1,\dots, k$. Both $Y^{(i)}$ and $X^{(i)}$ are vectors of length $n_i$. For each dataset we regress $Y$ on $X$, so we have $k$ ...
0
votes
1answer
42 views

Why lasso yield a higher mse then ridge?

I do a rige and lasso regression on a train data set and get the lambdas via cross validation and evalute the prediction ...
4
votes
0answers
39 views

Prediction with OLS better then prediction with lasso or ridge

I did a regression on a train data set with 7000 observations and 50 explenatory variables with ols ridge and lasso. The lambda was chosen via cross validation. After that i wanted to compare the ...
1
vote
0answers
29 views

My Neural Network gives a wrong prediction when I specify more nodes

I recently got interested in neural nets. After reading a bunch I tried to make one in python using numpy. I fed in some sample input and output data. When I train the neural net and and then ask it ...
0
votes
0answers
12 views

Prediction models and correlation in R [duplicate]

Forgive if I am not a seasoned statistician, I am starting out since some months. It's very difficult to starting out, I think you know it. I don't know many terms, and what it's super-difficult is ...
0
votes
0answers
20 views

Forecasting with two rank correlated data sets

I have a machine that produces widgets. I know the more widgets the machine produces the more heat it produces. When I observe the machine I see the following temperatures: [20,21,19,25,30,40,45,47,...
0
votes
0answers
13 views

R: different predictor p-value in linear and multi-linear models [duplicate]

Here is what I did: 1) I divided the mtcars dataset into a training set (80%) and a validation set (20%). 2) I built a simple linear model predicting mileage (mpg) based on displacement (disp). 3) ...
0
votes
1answer
30 views

R: different predictor p-value in linear and multi-linear models

Here is what I did: 1) I divided the mtcars dataset into a training set (80%) and a validation set (20%). 2) I built a simple linear model predicting mileage (mpg) based on displacement (disp). 3) ...
0
votes
0answers
24 views

Predicting future cash flows: forecast individuals vs. using averages

I am trying to find a good approach for a statistical problem, as well as a method for quantifying error with the approach used. Problem description: The task is to predict future cash flows for an ...
0
votes
1answer
45 views

Estimating values knowing their Pearon's r and their means and standard deviations

I apologize if my question is exceedingly simple. Imagine, for example, I am studying a paper which explains that two variables correlate with a certain Pearson's r (no p-value, no confidence interval,...
1
vote
1answer
54 views

Predicting relationships between variables from independent measures

I have a series of experiments where a certain set of parameters (let's call them P1, P2, ...) have been quantified in single cells. For technical reasons it is not possible to quantify all of these ...
3
votes
0answers
36 views

How to compare transformed and untransformed linear models?

I have a linear model which doesn't have any particular issues with its assumptions (diagnostics plots look well). However it has a slighly skewed response (skewness approx. 0.5) and few skew ...
0
votes
1answer
36 views

How to reduce multicollinearity between predictors

I would like to run an prediction model and have a set of continuous independent variables. They are all important but highly correlated. How can I effectively reduce collinearity and still use these ...
0
votes
0answers
16 views

Predict Next Purchase Order Value

Let's say we have historical customers order by value and we're trying to predict the next order value for each one of them. We don't have the date/time of past orders, and we don't care about the ...
1
vote
0answers
22 views

why LSTM word based model works better with random input seed than fixed input seed?

I have implemented an LSTM model that have 2 LSTM layers, a dropout layer and a dense layer for predictions. I trained my LSTM model on 1000 XML files. Each file has 4 main markups with very simple ...
2
votes
2answers
39 views

How do we compare count models for prediction and inference?

I have estimated a number of count models on a data, including Poisson, Zero-Inflated Poisson (ZIP), mixed-effects Poisson, mixed-effects ZIP and, a few different versions of each of these based on ...
0
votes
0answers
23 views

Suitability of fitting old model to new data for time series forecasting

What I usually do to forecast is: Train a model (i.e. bats, stl + arima|ets, ANN, ELM, etc ...) through a model function obtaining a trained model object, based on information criteria or cross-...
1
vote
1answer
24 views

Minimum number of observations per group in a prediction model

I ran a univariate logistic regression analysis on some risk factors for a dichotomous outcome. One of the variables is highly significant, but only has 5 outcomes in one of the groups. The confidence ...
1
vote
1answer
20 views

Dealing with Varability in Predicting Deaths

I hope this is the best place to ask a question like this and will not be to vague for this type of forum. I am hoping for some corrective guidance with an issue I am having. I work for a non-profit ...
0
votes
0answers
17 views

How to predict the probability of each binary event in the next day?

I'm now doing a project of transportation planning. In the cuurent step, I want to predict the travel demand in a new way, maybe using machine learning instead of traditional transportation model. I ...
0
votes
0answers
12 views

Strategy for sampling units from out-of-sample test set to validate predictions

Are there any strategies for thinking about how to sample units from an out of sample test set to validate predictions? For example, we are predicting outcomes for households and have trained models ...
3
votes
1answer
58 views

sklearn LogisticRegression only predicts 1, but predict_proba has many values

I am getting a strange output from sklearn's LogisticRegression, where my trained model classifies all observations as 1s. ...
0
votes
1answer
25 views

To report the fraction of explained variation, should I use R2, adjusted R2, or crossvalidated R2?

I am fitting a linear model (normal error structure) with several linear predictors. In R: lm(y ~ x1 + x2). I need to report how much variation in y my model explains. Which of the following ...