Predictive models are statistical models whose primary purpose is to predict other observations of a system optimally, as opposed to models whose purpose is to test a particular hypothesis or explain a phenomenon mechanistically. As such, predictive models place less emphasis on interpretability and ...
4
votes
1answer
154 views
Incorporating a treatment into a classification scheme
I have about 400 pieces of silver of different geometric dimensions. They were assigned to six groups and each group went through a series of stress tests, such as bending, pulling, putting in fire ...
6
votes
1answer
277 views
How can I generate predictions from the randomSurvivalForest package in R?
I'm trying to use the randomSurvivalForest package in R to predict the next event in a series of events (using ...
0
votes
1answer
137 views
Flexible discriminant analysis with discrete predictors in R
I am using the mda package and in particular the fda routine to classify in term of gear a set of 20 trips. I preformed a ...
2
votes
1answer
144 views
Predicting Football match winners based only on previous data of same match
I'm a huge football(soccer) fan and interested in Machine Learning too. As a project for my ML course I'm trying to build a model that would predict the chance of winning for the home team, given the ...
1
vote
1answer
100 views
How to validate and compare models predicting a binary variable?
I have a question about determining which models are "better" and how to assess that info.
Let's say I have three models, each which predicts our bid on won ping. Our bid is a continuous variable and ...
2
votes
2answers
76 views
What machine learning techniques can, once trained, generate prediction despite some missing inputs?
I have a training set where the inputs & outputs are all present, but I suspect that in the data where I want to do prediction, I will occasionally encounter scenarios where a small fraction of ...
2
votes
0answers
49 views
How does predictive model for the Eurovision Song Contest work?
I've encountered interesting prediction of Eurovision Song Contest http://mewo2.com/nerdery/2013/05/12/eurovision-2013-first-predictions/ it based on some kind of Bayesian model I assume but I don't ...
2
votes
2answers
45 views
Advice needed on auction system simulation
I am trying to simulate an auction system in which a number of competitors, $N$, independently offer a discount from a reference price previously published by the buyer.
The order is awarded to the ...
2
votes
2answers
260 views
Comparable salary in a different city calculations
I am doing a salary comparison depending on 2 cities. Very similar to this website
So you select a source city, then a destination city, your current salary in the source city and it will output the ...
0
votes
2answers
159 views
What statistics should I use for evaluating the accuracy of predictions?
I have two variables representing 1) players' predicted fantasy football points and 2) players' actual fantasy football points scored. What statistics are best for assessing the accuracy of the ...
0
votes
0answers
22 views
Should the number of parameters depend on the purpose of the model?
I am curious what are some arguments for/against increasing/decreasing the number of parameters depending on the purpose on the model.
I am currently building a model which will be used for ...
1
vote
1answer
117 views
Characterizing the inter-arrival time of software threads
For a multi-threaded application, I want to identify the nature of the application based on the arrival times of each thread. Example(Are thread launch spaced regularly, are they bursty in nature or ...
0
votes
0answers
47 views
When to Log/Exp your Variables when performing Linear Regression?
I'm doing regression using Random Forests for predicting prices based on several attributes. Code is written in Python using Scikit-learn.
How do you decide whether you should transform your ...
0
votes
1answer
166 views
Negative binomial — IRR interpretation for predictors
I have a zero-inflated negative binomial model. I have used incidence rate ratios and I'm trying to interpret the coefficients in relation to my predictors. Most of my predictors are continuous ...
1
vote
4answers
126 views
Measuring representativeness of a sample using covariates
I was provided with quite a small sample of labeled (variable of interest) observations to train a model to predict unlabeled observations. All the observations are associated with many covariates. ...
0
votes
0answers
12 views
Mean absolute percentage error (MAPE) in Scikit-learn
How can we calculate the Mean absolute percentage error (MAPE) of our predictions using Python and scikit-learn?
From the docs, we have only these 4 metric functions for Regressions:
...
1
vote
1answer
244 views
Predicting dependent variables where independent variables are samples with differing sample size
Is there a standard method of dealing with independent variables, where they are samples from a known distribution, and the sample sizes differ from sample to sample?
I'll give an example of what I ...
0
votes
0answers
53 views
Statistical tests on the revenue data of a small business
I have daily revenue data from a small business with 6 locations. The business sells food products that range from roughly \$2.00 to \$9.00, mainly to professionals. They do over a million dollars a ...
3
votes
1answer
41 views
How do I combine two predictors?
I am trying to classify a data set with 2 boolean values.
I have two classifiers that may/may not be independent. The first one is 65% accurate, and the second one is 60% accurate.
Can I combine ...
0
votes
0answers
29 views
Question about predictive bias - intercept and slope bias
I am slightly confused on how to determine a slope and intercept bias. I have an assignment where i am supposed to conduct a gender predictive validity bias analysis.
However, my lab handout and the ...
5
votes
2answers
81 views
Why do categorical predictor variables in regression need to be recoded as multiple predictors?
I'm learning about machine learning using Python's library scikit learn, and in their tutorial here they mentioned about a categorical variable color which can have ...
3
votes
2answers
104 views
Prediction and explanation of user rating based on multiple criteria
I'm trying to figure out a way to both predict how a user would rate a certain document, as well as an explanation of why certain documents are rated a certain way.
A user is represented by:
...
0
votes
1answer
66 views
Why is Hedonic Regression used instead of Linear Regression
Why is Hedonic Regression used (especially in housing prices) instead of Linear Regression?
There do not seem to be any libraries in Python (and R) for Hedonic regression, is it too niched a ...
1
vote
1answer
98 views
Modeling Outliers of Normal Distribtuion
I am using a linear model to predict under-nutrition in children under 5. The common metric discussed is stunting (a binary outcome) which is defined as being more than two standard deviations away ...
2
votes
1answer
109 views
Analogues of sensitivity and specificity for continuous outcomes
How can I calculate the sensitivity and specificity (or analogous measures) of a continuous diagnostic test in predicting a continuous outcome (e.g., blood pressure) without dichotomizing the outcome? ...
0
votes
1answer
49 views
Algorithms for regression analysis which can handle large scale datasets
I am a CS undergraduate student and for my final project i developed a regression algorithm that is suited for large-scale datasets (i wouldn't say 'Big Data', but still large scale).
For the final ...
4
votes
1answer
60 views
Predictive performance depends more on expertise of data analyst than on method?
I've come across a rumour that some study showed that the performance of predictive models depends more on the expertise of the data analyst with the chosen method than on the choice of the method.
In ...
2
votes
0answers
19 views
Predict binary occupancy vector from history of vectors
I have a set of binary vectors where each vector represents one day of occupancy in a house and consists of 48 elements (each element for 30 minutes of the day). Each element can be 1 meaning that ...
2
votes
1answer
37 views
How to report the results of cross-validation for comparing two models?
I want to compare the predictive power of two models. For this, I calculated the difference in some measure of predictive performance over many cross-validation replications. Now I have a distribution ...
2
votes
1answer
54 views
What exactly is the equation for SVM classification for new example?
I understand that in the case of Logistic Regression, we simply multiply our weights with Input example for classification. But what exactly is the equation that we calculate in the case of SVM to ...
2
votes
2answers
85 views
Medical Insurance Fraud Detection: Text analysis
I'm trying to analyse a dataset to detect fraudulent insurance claims. Unfortunately, other than basic demographics the rest of the claim is a free format OCR scanned text file made from documents ...
0
votes
2answers
47 views
In SVM, what are the labels and how do you get them from the data?
I'm working on a school project and have decided to use SVM for stock market prediction. I have a 1000x5 matrix of stock quotes containg data for open, close, high, low, volume data.
From what I ...
2
votes
1answer
121 views
Creating an estimator with varying shock levels (SD) in R?
First time posting!
I'm trying to create a logit estimator using a looping simulation, where the loop detects the number of correct prediction (my code is below). Is it possible to change the shock ...
0
votes
1answer
86 views
AIC, BIC, DIC, model selection criteria
I am trying to understand the difference between these parameters, and their application. Was hoping to get some correction/clarification to my statements. I have a training set and cross-validation ...
0
votes
0answers
38 views
Pre to Post Repeated measures for categorical dependent variable and more than one independent variables both continues and catgorical
I am doing Prenatal (before delivery) and Postnatal (after delivery) depression assessment (2 time points).
Dependent variable is Depressed/Non depressed mother on a measure of depression.
When as ...
1
vote
0answers
20 views
Hierachical Predictors in a Regression
Note: Mainly this question pertains to predictions from a model.
If the unit of analysis of a regression (or any predictive model really) is the individual retail store and these stores are organized ...
3
votes
2answers
99 views
How I can deal with too many variables in training a data set?
I am trying to train a predictive model on whether a given person is ( male or female) based on behavior cues we've obtained from online surveys.
The dependant variable will be a binary ( 1 or 0 ...
6
votes
2answers
103 views
Is there overfitting in this modellng approach
I recently was told that the process I followed (component of a MS Thesis) could be seen as over-fitting. I am looking to get a better understanding of this and see if others agree.
The objective of ...
1
vote
0answers
18 views
Calculating error bars for Excel Linear Regression [duplicate]
I've ben sent a forecast of sales from a consultancy. It uses Excel's LINEST function, taking 4 factors that seem to have affected sales in the past, and used them to make a prediction.
How do I go ...
2
votes
1answer
207 views
Figuring out probabilities with Hidden Markov Models
I'm really new to statistics so sorry in advance if this question does not make sense.
Background:
I'm trying to learn about hidden Markov models and they seem interesting but I was wondering about ...
1
vote
1answer
552 views
Regression technique for data comprised of categorical explanatory variables & a continuous response variable
i suppose one way to characterize data is by a combination of the variable types that comprises it:
...
2
votes
0answers
35 views
Modelling rainfall to size storage tanks
I have daily rainfall data for a given site going back about 30 years.
I have a building with an average daily demand for water of $L$ litres and a catchment of $A$ m$^2$ with a runoff coefficient of ...
2
votes
1answer
46 views
Re-scaling a confusion matrix after down sampling one class
Let's say I have a large, un-balanced binary classification problem (in reality nrow is more like 500k, and ncol is more like 500):
...
0
votes
1answer
43 views
Model assumptions not met but model has predictive capabilities
This is a general question, has probably been asked before (I searched and did not see similar), may have a simple answer, may be ridiculous, and is just to satisfy my own curiosity.
Say we build a ...
4
votes
2answers
71 views
Predicting chemical property (Boiling Point) from a SMILES string
I was trying to develop a model for predicting Boiling Points (BP) given a chemical name. One good and unique (ok, almost) way to encode a name is the SMILES notation string. The details of the ...
0
votes
0answers
29 views
Various models not improving basic rpart result
I have a data set with 10,000 or so samples in it and 100 or so features. I've created a training set and test set and am trying to predict a numeric value. I've used rpart to determine the most ...
0
votes
0answers
74 views
Last value in sequence for known trend — is this an app for nonlinear regression?
I would most appreciate names of methods/techniques. Again, I don't have the terminology to describe the problem very well -- I'll edit as needed.
There is a variable (x,y,z) where x is a timeframe ...
0
votes
0answers
43 views
Weighting and time series with machine learning
I'm trying to produce a model to predict the price of a product on the basis of several factors effecting previous time-stamped sales. I am certain that older sales are less relevant to the prediction ...
5
votes
0answers
373 views
Sample size and cross-validation methods for Cox regression predictive models
I have a question I would like to pose to the community. I have recently been asked to provide statistical analysis for a tumor marker prognostic study. I have primarily used these two references to ...
3
votes
0answers
38 views
Modeling pass rates for departments and courses within a school
Suppose I have a regression model, for example a logistic regression model, which provides a score between 0 and 1 reflecting whether or not that a student will pass a course given certain variables:
...
