Questions tagged [predictive-models]
Predictive models are statistical models whose primary purpose is to predict other observations of a system optimally, as opposed to models whose purpose is to test a particular hypothesis or explain a phenomenon mechanistically. As such, predictive models place less emphasis on interpretability and more emphasis on performance.
3,156
questions
0
votes
0
answers
10
views
Can the first latent variable of a PLSR model explain less variation in either X or Y compared to the second latent variable?
I currently have a n x m dataset as the X block and a n x 1 dataset as the Y block. I am using the ROPLS package in R, and I've noticed that there are times when R2Y is greater in the second latent ...
- 113
0
votes
0
answers
11
views
Forecasting at areal level from repeated cross sectional survey with binary measurements with spatiotemporal correlation
Problem statement
I’m having trouble understanding and expressing my options for formulating a model in “statistician language” in order to achieve my goals. I’m hoping this community can help me with ...
- 141
1
vote
0
answers
23
views
Logistic regression with a lot of predictors/ general understanding
i'm currently planning to write my bachelor thesis but it's been a while since my last statistics seminar, i'm extremely rusty and so any guidance here would be appreciated.
I have a relatively small ...
- 11
0
votes
1
answer
20
views
How do I classify certain input elements in training my model? [closed]
I have a data for a ton of tennis players & I want to train my model to understand each of the different players are individual entities. Then, I would like to input 2 players and have my model ...
- 1
6
votes
2
answers
388
views
Re-selecting best sales forecasting model each month. Is this overfitting?
One of our teams works on sales prediction, and they run ~10 models each month for each product (+100). Then they use the best fitting model for next period prediction, which may be a totally ...
- 595
3
votes
1
answer
32
views
Abnormally large confidence interval with binomial gam when p->1 at max of predictor range
I am running a gam (mgcv in R) to model a non-linear effect of time on a binomial reponse (positive or negative sample). This is a minimal example of such a model:
...
1
vote
0
answers
22
views
How to derive conditional posterior predictive distribution from definition of posterior predictive distribution in bayesian regression?
In my situation, I have a set of data points:
$$ z_{0:n} = \\{ (x_0, y_0),\dots ,(x_{n-1}, y_{n-1}) \\} $$
I am trying to figure out how to derive the fully expanded form for the conditional posterior ...
- 379
1
vote
1
answer
30
views
How can I find what is driving the change between my updated model and the original model?
I'm working on updating a model of rents (which is currently simple OLS) for my employer who has a large national portfolio. By tweaking here and there and drawing in a large amount of exogenous data ...
- 11
0
votes
0
answers
12
views
Check_model performance package interpretation plots
I am fairly new to r and statistic and I am building GLMs for frogs occupancy and abundance using a dataset with 57 observations and 13 independent variables. As some variables are correlated the ...
2
votes
0
answers
17
views
Force GLM through zero [duplicate]
I am working on a glm model using a binomial distribution. I want to force the intercept through zero, as I know that biologically this makes sense. I have used the formula to ...
1
vote
1
answer
31
views
Assumptions in definition of "Log Pointwise Predictive Density" (LPD)
In Equation 2 of Vehtari (2016), the log pointwise predictive density is defined as
$$
\text{lpd}
= \sum_{i=1}^n \log p(y_i | y)
= \sum_{i=1}^n \log \int p(y_i | \theta) p(\theta | y) \text{...
- 21
0
votes
1
answer
34
views
How can I improve R2 score in my regression model? Predicting House Prices
I have trained some data on a House Pricing dataset.
and I'm getting a not-so-bad R-2 score of nearly 0.5 as you can see below:
I wanted to ask how can I improve this R-2 Score and get more precise ...
- 1
0
votes
0
answers
18
views
How can I assess internal validation (discrimination, calibration) of a Fine-and-Gray competing risk model, fitted using a MFP-algoritm in Stata?
I'm a post-doc at Karolinska Institute, and I'm working on developing a Fine-and-Gray competing risk model to predict endometrial cancer recurrence/progression with death as a competing event.
I have ...
- 81
0
votes
0
answers
8
views
nlme prediction differs largely from a 3 way interaction model to its post-hoc follow up
I'm trying to predict that speed at which people complete a walking test, where they perform this test for multiple trials and overall increase their performance on each trial. They perform as many ...
0
votes
1
answer
25
views
Predicting binary outcome when predictor variable increases
Suppose I have a simple dataset of numerous observations, each with a continuous numerical variable $x$ and a binary numerical variable $y$ (with values 0 for unsatisfactory, 1 for satisfactory).
How ...
- 1
0
votes
0
answers
10
views
Naive Bayer Classifier - Do we use dependence structure?
If we apply Naive Bayer Classifier and predict an unseen observation just by using the posterior probability calculated with Bayes theorem combined with the naive feature independence assumption, do ...
- 435
2
votes
0
answers
46
views
Chi-Squared for demonstrating confounding in Logistic regression (or not...)
I am using logistic regression for inference and classification, using data from 190 X-rays/subjects. We want to see if certain X-ray measurements could predict development of a disease (Case vs ...
- 41
0
votes
0
answers
7
views
Pooling survey years in random forests
I am currently trying to predict household wealth in a random forest, using survey data.
My problem is the number of the observations. I have the option to either choose single years (n=4.000) or ...
3
votes
1
answer
54
views
repeated measures mixed effects model with time-varying covariates in r
Assuming that we have longitudinal data on pulmonary fibrosis with some patients undergoing transplant while others received medical treatment. Each patient is represented by many rows depending on ...
- 241
0
votes
0
answers
17
views
Survival analysis: Use of "legible" metrics like RMSE
I apply a Cox proportional hazards model to some machine failure data and I want to know, "how good" my model is. Metrics like RMSE or MAE are said to be not suitable for this kind of model, ...
- 611
1
vote
2
answers
89
views
Interpretation of posterior predictive distributions
QUESTION UPDATE due to the comments I have received so far.
The data, the example and the results below are fictitious, as I am interested with the correct interpretation of these results.
Suppose I ...
- 73
0
votes
1
answer
24
views
Trading card game poss
A trading card game that is called Flesh and Blood (description and rules here) has two players construct a 60 card deck and the hand limit is 4. Player 2's Deck color combination: 60 cards in deck; ...
- 1
0
votes
0
answers
27
views
Do I use $P(y_t|y_{1:t-1})$ instead of $P(y_t|z_{t},y_{1:t-1})$ for prediction in a Hidden Markov Model?
I am confused what to sample for getting the prediction $y_{t}$ if I have access to the previous observations $y_{1:t-1}$ and the hidden states $z_{1:t}$.
I want to predict the observation at time $t$ ...
- 777
1
vote
1
answer
36
views
How does the predict() function used with GAMs work with generating values from original data? [closed]
Hi I'm wondering if I may gain some insight on how the predict() and predict.gam() function work.
Below I have my GAM model and the name of my original dataframe I am using.
...
- 21
0
votes
0
answers
13
views
How to perform random walk on multilayer network to predict new edges
I have a multi-layer network that is a union of 3 networks (field of human biology/ Omics data). The 3 networks have dense connections within each other (local), however sparse connections to each ...
- 21
0
votes
0
answers
8
views
Better Prediction model with large DIC value
I am using Integrated Nested Laplace Approximation (INLA) to predict birds population.In my (INLA) models, I've noticed that when I include auto spatial correlations, my predictions are good and make ...
- 141
0
votes
1
answer
22
views
Exponential survival model using psm in rms package
I would like to run an exponential survival model using the psm function in rms package, but I am still trying to understand the ...
- 215
1
vote
0
answers
12
views
Model choice based on test/train/validation split [duplicate]
My question is very simple, but no matter where I look it up, it seems that I get another answer.
Take a simple classification task. Let's say I trained a kNN, LDA and logistic regression on it for ...
- 435
0
votes
0
answers
16
views
Lasso regression test MSE lower than train MSE
Im currently using Lasso to build a predictive model for numeric variable .
Before scaling the features I split the data for train test and validation . I have a feature named 'year' and i wanted the ...
0
votes
0
answers
6
views
Modeling a population over time given certain parameters
I am trying to model a population over a span of time. The span of time is between the birth of the first person and the death of the last one, say the years 1867 and 2151. Total people born is a ...
0
votes
0
answers
12
views
Consequences of maintaining IID assumption for prediction model training, but relaxing it for model testing
Let's say you're developing a prediction model, and you are confident that your data are IID. For example, you have a dataset where each row represents a different patient, and you build a model to ...
- 1
0
votes
1
answer
58
views
Constructing a one-sided hypothesis test for joint probabilties of negative binomial distributions
I am conducting research on Codling moth population/trap capture models. The end goal is to have a hypothesis testing model that will provide whether or not (at some significance level $\alpha$) the ...
- 101
0
votes
1
answer
25
views
Find event date given the probabilities of finding an event
I have a set of clinical notes with dates for each patient and an NLP models which gives a score between 0.0 and 1.0 of a certain event being present in the note. Given the scores, what is the best ...
- 101
0
votes
0
answers
46
views
Why use integrals when building exponential composite functions?
I recently read this paper, which describes a generalisation for statistical exponential decay models used in ecology. Essentially, the parameter $k$ of the exponential decay function $f(x) = ce^{-kx}$...
0
votes
1
answer
34
views
How to create a composite variable from survey data?
I'm working with survey data, which I've never done before (from the SHARE database). What I basically want to do is create a composite variable for cognitive function, mental health and maybe some ...
1
vote
0
answers
19
views
Modeling Wildlife Survey Data with Conditional Outcomes
I'm working on a project involving surveying for an invasive species which is removed upon encounter. When an individual is encountered, morphometrics are collected (e.g., size and sex). I'm ...
- 25
1
vote
0
answers
29
views
Prediction interval as per probability
I am practicing the regression problem using the sci-kit learn dataset. The dataset is about housing prices. When we use a regression model, it predicts a number. Based on the predicted value and ...
- 11
0
votes
0
answers
14
views
Extrapolating/forecasting treatment effects from difference-in-difference model
My goal is to extrapolate or forecast dynamic treatment effects into the future using a fitted model. My data consists of two groups (treated and control), seven time points, and a continuous outcome.
...
8
votes
2
answers
535
views
Why does the normality assumption not affect Linear Regression in large samples?
I've read once that the normality assumption shouldn't be a problem and that you actually shouldn't care that much if your sample is large. Why is that? Can someone give me a mathematical explanation? ...
- 827
0
votes
0
answers
16
views
How to Evaluate a Single-Value Prediction for a 6-Month Period Against Historical Data?
I'm tackling a time-series forecasting issue with daily granularity, aiming to predict a single aggregate value that represents the total sum of incidents over a 6-month period. My approach involves ...
- 57
0
votes
0
answers
40
views
Application of mixed-effects model for unbalanced sample size and repeated measures
In my experimental design I have 4 treatments, 3 replicates per treatment and 3 blocks. In each plot I measured whether a plant is infested or not ("Infestate" variable). This measure has ...
- 67
2
votes
1
answer
72
views
models MIMIC, predicted probabilities?
I am investigating MIMIC (Multiple Indicators Multiple Causes) models since with them I can do regressions, including factors (made up of several items), and the observed variables (glycemia, ...
0
votes
0
answers
5
views
Statitical comparison of regression prediction to sample of data
I have a dataset, let's call it data1(x,y), where the data y is measured at 16 values of x. I use this dataset to fit to a 3-parameter model. I can obtain estimates for the three model coefficents ...
- 1
0
votes
1
answer
37
views
non-significant p value in a multivariable cox regression following exhaustive model selection
I run an exhaustive model selection for Cox proportional hazard in R using "glmulti" package. I used the best model for creating multivariable Cox regression. In the multivariable Cox hazard,...
0
votes
0
answers
10
views
Removing observations with missing target values in the test set
I'm building my first predictive model and seem to be having a fundamental confusion about missing target values.
I'm predicting treatment outcome (with both regression and classification methods for ...
- 115
3
votes
1
answer
67
views
What metric should I use for a Regression model with a gamma distributed target?
Background
I'm building a regression model on insurance data to predict the losses associated with a policy. I'm running an Optuna optimisation function to help me with this, but I'm struggling with ...
- 625
3
votes
0
answers
183
views
Recreating data variance from the posterior distribution
Recreating data variance from the posterior distribution
Take a set of data points $(x, y)$ with (Gaussian) uncertainties $\sigma_y$ on the $y$ coordinate; they are modeled as $y \sim f(x; \alpha) + \...
- 131
0
votes
0
answers
36
views
Log transformation uses
I am trying to understand how the migration of a male member affects the number of hours spent by left-behind women in various agricultural and non-agricultural activities. I used a simple OLS model ...
- 1
0
votes
0
answers
28
views
Daily monitoring of churn prediction model
I've written and trained a churn model that is scheduled to run every day and make new predictions for the probability of each customer to churn within coming 365 days, from the day the scoring is ...
- 363
2
votes
1
answer
41
views
R=X*Y is the relationship. Is predicting R and X and obtain Y same as predicting X and Y to obtain R?
Of course the numbers will be different, I mean more in terms of relationship. I know that X affect R and Y affects R . X and Y are independent but since R is a product of X and Y , I dont think that ...
- 61