Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [prediction]

Prediction of unknown random quantities, using a statistical model.

0
votes
0answers
7 views

How to build an interval around a final Logistic Regression classification prediction?

I have a data set of 5000 hotel reviews. My boss wanted to know how many of the reviews were 'negative', so I set out to build a model. I began by tagging 200 of the reviews 'positive' or 'negative'. ...
0
votes
0answers
17 views

Estimating probability density for forecasts

I've used a handful of algorithms for forecasting future values in a time series. But sometimes what I'm really interested in is not the predicted value, but the probability that some future will be ...
0
votes
0answers
10 views

Nested cross-validation for performance assessment and simple CV for model building?

I am not sure if I understood Dikran Marsupial in this thread correctly: Dikran Marsupial comment on Jan 22 2019 at 11:38: Dikran suggests that nested CV is only used for performance estimates of ...
0
votes
0answers
6 views

Averaging GLM's based on delta AIC < 2 and using the resulting coefficient estimates with their associated raster to build a predictive raster

I'm trying to construct a predictive spatial raster based on averaged GLM's Not the exact code but an example: mod1 <- glm(c~ x1 + x2 + x3, data=data) mod2 <- glm(c~ x1 + x2, data=data) mod3 &...
1
vote
1answer
25 views
+50

Somers' D for model validation

Somers' D as defined for example in "The predictive accuracy of credit ratings: Measurement and statistical inference" by Walter Orth is defined for the case when Y is predicted by X as $$ D_{XY} = \...
0
votes
1answer
13 views

Using one-hot encoded features along with continuous-valued features?

The task I wanted to do is a prediction task where most of the features are continuous numbers and some of the features are one-hot encoded. I am training a neural network and I wondered that, is it ...
0
votes
0answers
10 views

Predication of MDN

How to get the prediction of Mixture Density Networks? In MDN one models the conditional density: $P(y^i |x^i) = \sum_{j=1}^{m} \alpha_j(x^i)\phi({y^i|x^j})$, so I guess one just sample from the ...
0
votes
0answers
17 views

Which variable to predict when using Gaussian Processes: $y_*$ or $f_*$?

We have the following model: $y_t=f(x_t)+\epsilon_t$, where $f$ follows a gaussian process, and $\epsilon$ a normal distribution. Which quantity should I predict, $y_*$ or $f(x_*)$? The difference, ...
1
vote
1answer
53 views

Missing data when actually predicting - Additional model legit?

This is a theoretical question, but I have already stumbled upon this issue a few times: My learning data are not complete, but I manage to handle the missing values. Now it's time for actually ...
0
votes
1answer
22 views

Estimating potential moves in a time series relative to an other

Say we have two time series which over time have a strong correlation, say above 0.8, say for example the price of oil and share price of an oil exploration and extraction company. Now say that we ...
0
votes
0answers
8 views

Cross Validation with Panel Data of Variable Length

I have a problem of wanting to predict $y_t$ from $x_t$ and some lags given a series of $\{(y_t, x_t)\}_{t=1}^T$. To build my prediction algorithm I was hoping to use cross validation techniques such ...
0
votes
0answers
11 views

How to validate PLS2 with R?

I have a PLS1 modell wich predicts single chemicals. Now I would like to run a PLS2 modell to predict two or more chemicals from a mixture. I programmed it in R and the modell runs, but i don´t find ...
0
votes
0answers
12 views

How the probability in a decision tree is calculated? [closed]

I build a classification tree and predicted the probability that an observation belong to the positive class. Now i ask myself how this probability is calculated?
1
vote
1answer
128 views

Fitted RF: difference between the probabilities in $votes and predict (type=“prob”)

Say we have a data frame df where diagnosis is the first column. There are only 2 possible values for ...
0
votes
0answers
16 views

what test is suitble to predict seen and unseen data

In my model, I want to predict both seen and unseen data and get the result that maximizes the accuracy. The problem is that in some cases seen data are well predicted since the model is overfitting, ...
0
votes
0answers
9 views

Forecast using Cross Sectional data? Correlation vs Causality

I try to predict the revenue of a store given its characteristics. Is it possible to establish a parsimonious model that can predict at a different point of time without establishing a logical or ...
1
vote
1answer
31 views

Baseline hazard function is the hazard function obtained when all covariates are set to zero

I am trying to learn Cox proportional hazard model but I have hit a wall with the basehaz function. Lets suppose for example I have some data that I want to use ...
1
vote
1answer
23 views

How to choose a model for survival regression when data does not fit assumptions?

I am trying to perform survival regression (prediction) on a dataset of lifetimes, which is highly concentrated around 1, with a significant right skew. The below photo is how it looks when log-...
1
vote
1answer
19 views

Which cross-validation measure of model fit performs best when the objective is probability estimation in classification tasks?

Suppose a binary outcome $Y=0$ or $Y=1$ where $P(Y=1|X)=f(X)$ is a function of $X$. The goal is to estimate $f$ as closely as possible using a classifier that returns a probability estimate (e.g. ...
0
votes
0answers
18 views

AdaBoost - get prediction for the specific number of estimators

I use AdaBoostClassifier from sklearn.ensemble. I trained my model using 1000 estimators: ...
1
vote
2answers
39 views

Does the MSE values of regression coefficients sum up to the MSE value of the regression model in which the regression coefficients are included?

I think either i dont understand something or i try to mix something that are different things. The mse value of a regression coefficients tells me how good i estimated the coefficent. Does it mean ...
0
votes
0answers
15 views

How to predict the complete Ranking (1-6) of six Machines. (Multi target prediction)

I would like to predict the "Ranking" (1-6) of all 6 coffeemachines with the Big Five personality traits (OCEAN) and i don't really know which Algorythm would be the best for this task. Any ...
3
votes
0answers
52 views

Does anybody know this measure of model fit / prediction error?

Let $y_i$ be the true value and $\hat{y}_i$ a prediction from a model. Then, for example $$B=n^{-1}\sum_{i=1}^n \hat{y}_i - y_i$$ is the prediction bias and $$MSE=n^{-1}\sum_{i=1}^n (\hat{y}_i - y_i)^...
2
votes
1answer
42 views

How to determine what type of trendline to use on measured data?

I want to put acoustic panels in a space to help reduce reverberation (RT60 is a measurement of how long it takes sound to decay by 60 dB). I have already calculated how many panels should ...
2
votes
2answers
110 views

How to use a cross-validated model for prediction?

I want to do the following steps: Train a model with cross-validation Use that model for future predictions (including my test set) cross_val_predict only gives me its predictions for the training ...
0
votes
0answers
18 views

Why i get the same MSE value for two least square models that differ in one explanatory variable?

I have two ols-regression models that just differ in one variable. It means that one model have the same variables like the other plus an explanatory variable more. I estimated both models on a train ...
1
vote
0answers
19 views

Heavily overlapping data

I have the data: tibble(r) %>% head() ...
0
votes
0answers
19 views

What model to use to predict multiple binary time series

I have dataset similar to above one. It contains ~ 1 lakh observations and and ~ 30 columns (those are my columns). I want to predict whether a particular individual is likely to watch television in ...
0
votes
0answers
18 views

conversion prediction / ROC curve using markov chains for channel attribution

I am currently working on a project on multi-channel attribution, using the channel attribution package from Altomare & Loris (2018), which uses markov chains for attribution. A walk-through of ...
0
votes
0answers
13 views

support vector regression predictions range

I don't figure out if this is a problem or not. My aim is to predict a continuous variable with a support vector regression. I use the sklearn python library. I'll go directly to the code. After the ...
0
votes
0answers
25 views

Predicting a seasonal signal

Consider a noisy signal $X$ and a time variable $T$ as seen in Fig. 1. This is a simulated generalization of data I am currently facing with the goal of making predictions for future time points $t_f$....
1
vote
2answers
50 views

Is the accuracy equal if the true positive and false positive rates are equal between two groups?

I was reading the paper "Equality of Opportunity in Supervised Learning" (link). In that paper there is a feature $A \in \{0,1\}$ and a binary outcome $y \in \{0,1\}$. The population is divided into ...
2
votes
1answer
55 views

Can balanced accuracy be higher than accuracy?

I have classification tree where the balanced accuracy of the test set is higher than the normal accuracy. I thought balanced accuracy can only have at his maximum the same value as the accuracy not ...
2
votes
2answers
57 views

How to check if i have strong linear relationship between dependent variable and independent variables in linear regression (OLS)?

I want compare the out of sample prediction from an linear regression model (OLS) and a regression tree. I read that OLS outperforms regression tree if the relationship between the dependent variable ...
-1
votes
0answers
58 views

What to do if random forest still overfit after grid tuning?

I have a random forest and an ols regression. Both models i want use for an out of sample prediction. Before tuning the parameters of the random forest the default settings of the random forest yield ...
0
votes
1answer
38 views

Is quantile regression a special case of OLS?

Quantile regression is often advertised as a way of "predicting change in the dependent variable that is not the mean." It seems like one can do this with linear regression, however. Am I correct? ...
0
votes
0answers
20 views

strong and independent extra explanatory variable doesn't improve linear regression

So I already have a linear regression on 3 predictors $Y = X_1 + X_2 + X_3$. Now I have an extra predictor $X_4$. Before I put in $X_4$, the original predictor using $\hat{Y} = X_1 + X_2 + X_3$ has ...
1
vote
0answers
22 views

What is the relationship between minimizing prediciton error versus parameter estimation error?

With the advent of statistical learning techniques, people are talking a lot about prediction error, while in classical statistics, one is focusing on parameter estimation error. What is the ...
1
vote
0answers
32 views

Does bias in regression coefficients affect the prediction?

Goal is to create ols model for out of sample prediction for log(wages). Theory say I could have a sample selection bias. So I choose the heckit method to correct for it. The correction term lambda (...
1
vote
0answers
26 views

Do unbiased regression coefficents yield better prediction?

I ask myself if a have a omitted variables bias in my regression modell the coefficients of the model are biased so the mse growth because this coefficents are biased right? So does it mean if i ...
0
votes
0answers
18 views

Kaplan Meier Diagnostic Utility

I'm trying to understand a paper that claims to have identified a gene expression signature that can distinguish primary from metastatic tumors. The authors stratify their data into patients with and ...
0
votes
0answers
33 views

Using the Standard Error of Prediction in the presence of practice effects

I’m wondering about the following hypothetical scenario. There’s a student who previously scored 40% on an examination with a pass mark of 50%, a mean mark of 60%, SD of 10 and a reliability of 0.6. ...
1
vote
3answers
49 views

Can a prediction be better with insignificant variables than with only significant variables (or none at all)?

I have two OLS models and want to do an out of sample prediction for wages on a test set. In the first model I excluded the insignificant variable. The second model has the insignificant variable. The ...
1
vote
0answers
28 views

How to model a specific distribution using domain knowledge rules [closed]

Suppose I have a variable Y that I want to predict with a model using predictor variables X1, X2 and X3. I have a large set of Y-data and from this I know with some certainty and accuracy the ...
0
votes
0answers
22 views

How can the balanced accurcay be bigger than the normal accuracy in unbalanced test data? [duplicate]

I constructed two binary classification tree's on two different training set's that i balanced with oversampling and undersampling. The test set is still unbalanced. After that i computetd the ...
1
vote
1answer
26 views

Confidence intervals for predictions from linear discriminant analysis

I wan't to draw 95% prediction area of an LDA model. I can draw the prediciton area, however with no information on the confidence. ...
3
votes
2answers
98 views

Suggestions on Modeling Approach to Model Percent Complete of a Task

I am trying to predict what percentage (or proportion) of a task is completed by various workers, given the time left until the deadline to complete the task and I'm looking for help on how to ...
0
votes
0answers
12 views

Predict a field based on grouped data

I am very new to ML and I have a requirement that consists of predecting the value of a field based on grouped data. Basically, I have a dataset that I need to split into classes. Each class is the ...
0
votes
0answers
20 views

Is my model selection procedure problematic for inference?

I'm not sure if this is "step-wise" model selection, but here is what I'm doing Decide a handful of models through exploratory data analysis. Fit the models to the data, and calculate their AIC. Pick ...
0
votes
0answers
63 views

EDIT) Bayesian prediction using regression

I have a very basic, introductory statistical background but learning Bayesian analysis with Bayesian Data Analysis(A.Gelman) and I desperately need any hint to help me grasp a concept. As long as I ...