1
vote
2answers
118 views

Logistic Regression Model Validation

I am validating a logistic regression model. This is the first time i am validating a model. I am using split sampling method. I have split data randomly into two parts - 70% development and 30% ...
0
votes
0answers
14 views

Validating a multivariate categorical model

I assume that my population is a sample of an unknown multivariate categorical distribution $\mathbf{X} = (X_1, X_2, \ldots, X_k)$. From this population, a sample $\mathbf{X^*}$ is available, I assume ...
0
votes
0answers
50 views

holdback validation - test and validation equal but training much better - acceptable?

I have a dataset that, after modelling (with bootstrap aggregation of trees), I find I have a very high and thus over-fitted training result, but my held-back validation and test portions are ...
2
votes
0answers
137 views

What is the criterion of model validation?

Suppose I have a model that boasts given 1 year's daily data to calibrate the parameters, it could predict the behavior of future 1 month. What should be the criterion for back-testing or, model ...
0
votes
0answers
341 views

How to write a loop in R to select multiple regression model and validate it?

I would like to run a loop in R. I have never done this before, so I would be very grateful for your help ! I have a sample set: 25 objects. I would like to draw 1 object from it and use it as a ...
2
votes
1answer
59 views

Resources for developing prognostic index-scores

I am looking for online resources or books explaining how to develop prognostic indexes-scores step by step. I am mainly wondering about model validation and transformation of regression coefficients ...
5
votes
1answer
114 views

What do you do with your testing data?

Suppose you perform a parameter selection study in an environment with not-so-many data points. You divide the available data into training and testing set and you plan to validate the future model on ...
2
votes
0answers
94 views

Validation of data partitioning

I've some numerical sequence of data monitored over time $v(i)$, which belongs to one of two classes, 0 or 1. I monitor the variable several times, obtaining a sequence associated to 0 or 1. I use ...
1
vote
1answer
118 views

Difference between Step validation and Cross validation

I'm trying to model some chemical data from a series of spectra. I've gone through the preprocessing step and selected "autoscale" (I'm using Pirouette). And when it comes to validation I'm a little ...