The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
1answer
10 views

Is the “angle=arccos(spherical)” scoring rule strictly proper?

The Spherical scoring rule is known to be strictly proper. However, it is not very intuitive. It's arccos, however, is the angle between the prediction ...
5
votes
1answer
82 views
+100

Why does Lucene IDF have a seemingly additional +1?

From the Lucene docs $\text{IDF} = 1 + \log\left(\frac{\text{numDocs}}{\text{docFreq}+1}\right)$ In other references (i.e. wikipedia), IDF is typically calculated as ...
0
votes
1answer
11 views

Is there a distance algorithm similar to Jaccard distance that handles scalar data?

we have the characteristics and (scalar) values of those characteristics for three (or more) people: ...
0
votes
0answers
19 views

I observed very different feature scoring from two different classifiers. What does it really mean?

Here what I've done. Given the dataset, I run a Random Forests and Logistic Regression with 5 Fold Stratified Data Sampling. Then I plot the feature importance for Random Forests and Logistic ...
0
votes
0answers
13 views

What is the “general convention” for coding the response variable in credit scoring models?

I would like to receive suggestions on what is the general (most common / popular) approach for coding the response variable in scredit scoring models (logistic regression) to denote "bad"/"good" ...
0
votes
0answers
10 views

Cronbach's alpha

I want to use the EAT-26 in my study , and the zero frequency is high in scoring because more than one answer represent zero in scoring ( never=0,rarely=0,sometimes=0) , so Cronbach's alpha is low ?! ...
0
votes
0answers
14 views

scoring/predicting for new observations

I have two data sets of variables where one of them - the new observations - has no dependent variable. The data set without a dependent variable has around 20 times the number of records. ...
0
votes
0answers
26 views

Regression: Predicting values over orders of magnitudes - what metric(s) to use?

What is a good metric to use for predicting values over several orders of magnitude? I can use R^2 but other measures like mean absolute error or mean squared error are pretty meaningless. In my case ...
2
votes
1answer
605 views

Stepwise Model Selection in Logistic Regression in R

I'm implementing a logistic regression model in R and I have 80 variables to chose from. I need to automatize the process of variable selection of the model so I'm using the step function. I've no ...
0
votes
0answers
48 views

Scoring model/formula to rank based on accuracy of data sources

Let us say i have 5 different data sources. My aim is arrive a score to arrive the accuracy of this data sources. It is like below 1. Refer the section Data confidence calculation in ...
1
vote
1answer
36 views

Cut Score Determination in IRT

Cut score or cut point is a minimum achievement or ability that examinees should get in order to get a 'Pass'. In CTT, MCQ scoring in the form of percent-correct scores are applied. Let say we ...
1
vote
1answer
65 views

optimal down payment estimation in credit scoring

Knowing I can estimate the risk of default, via logistic regression, of a consumer on a small loan... what would be the best way to estimate the optimal down-payment amount to ask for in order to ...
5
votes
1answer
171 views

Justifying and choosing a proper scoring rule

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
0
votes
0answers
19 views

How can I evaluate binary response models that have weighted observations?

I'm working with a binary response data set, but the importance of each observation varies over a factor of 100. Models to fit the data can accept a weight for each observation. But when it comes time ...
0
votes
0answers
37 views

Scoring function for categorical data

I would appreciate guidance on the following problem. There are three sets of urns, Set 1, 2, and 3. Each set contains the same number of urns, Urn 1, 2, 3. Each urn contains some number of Red ...
0
votes
0answers
19 views

Missing value replacement in modeling and scoring

Here I have two questions I build a logistic regression model. While building model I have few observations have NA values, so I replace with mean value. Model is looking good and when we tried to ...
1
vote
0answers
261 views

Bootstrapping in SAS - PROC LOGISTIC - Next steps ? how to score / perform diagnostics?

My question is as follows. I am referencing the following paper by David Cassell - wherein David talks about bootstrapping techniques in SAS using PROC SURVEYSELECT (many thanks to David - truly a ...
0
votes
1answer
181 views

Survfit function in R to score a new dataset

I have built a cox proportional hazards model in the R survival package. I want to score new data set using this model. I thought the survfit function would doing this using survfit(original model, ...
2
votes
1answer
73 views

Using proper scoring rule to determine class membership from logistic regression

I am using logistic regression to predict likelihood of an event occurring. Ultimately, these probabilities are put into a production environment, where we focus as much as possible on hitting our ...
1
vote
0answers
41 views

Proper scoring rules for observations with different supports

Suppose to have a bivariate variable $z_t=(x_t, y_t)$ indexed by $t=1,2, ..., T$. Suppose now that the two components have different support, i.e. in my specific problem $x_t \in \mathcal{S}$, where ...
0
votes
1answer
367 views

What data mining/machine learning approach to use for a scoring model?

Suppose I have a large data set with lots of features(attributes). And I'm tasked to build some kind of scoring model to rank certain objects with all these features. How do I go about doing this? ...
0
votes
0answers
18 views

Web page scoring function based on readability (LIX)

I'm creating a scoring system for web pages based on their level of readability using LIX. A selection of pages from a web site are crawled and given a score. Besides average and median, what ...
11
votes
2answers
225 views

Intelligence Squared Scoring and Winner Determination

There is an NPR podcast called Intelligence Squared. Each episode is a broadcasting of a live debate on some contentious statement such as "The 2nd amendment is no longer relevant" or "Affirmative ...
1
vote
1answer
47 views

Algorithm for scoring co-varying traits

I am sure this has been done, but I can't find quite the right approach. EDIT: Trying to explain better. The rows of colored boxes below are columns of molecular sequence data -- positions in a ...
4
votes
1answer
71 views

Relation between Scoring rule and Loss function in Parameter estimation and model selection?

Initially, I had only heard of MLE and use it for almost everything, e.g. point estimate and model selection (with some penalty). Then, MSE appeared, which seems to play the same role as MLE does. I ...
0
votes
0answers
37 views

Creating a model to interpret numerical scores

Good morning/afternoon everyone, first of all thanks to all of you for the valuable insights provided. I will be oulining here my current challenge, trying to provide as much detail as possible. ...
1
vote
0answers
22 views

Is there a reason to score on subsets of your data?

If you're training a model using cross validation on one set of data, and then you're scoring that model on a separate set of data, is it advised to score, say, 1000 times on subsets of your data? ...
1
vote
0answers
252 views

Random forest “certainty / likelihood score” - how to score records in RF mode in R?

My question is similar to this link Creating a "certainty score" from the votes in random forests? I am trying to build a random forest for a binary response (1 & 0). Let's say we have ...
0
votes
2answers
51 views

How to get an “averaged” score?

Assume a game wherein a character's "power" is made up of several factors, like speed, weight, body build, etc. And let's say that each of these factors were scored 0, 1, 2 wherein 0 means average, 1 ...
2
votes
1answer
237 views

Cluster many thousands observations (mixed variable types). Cluster subsample and then classify the rest observations?

I'm trying to run a cluster analysis on a large dataset (70k+ observations to cluster) with mixed variables (numeric, ordinal, binary and nominal). I don't think I can create the distance matrix using ...
3
votes
1answer
73 views

Deploying survival model for production

I've recently built a survival model with time-varying covariates (assuming equal time periods) using R and I am now in the process of putting this into Oracle for production using just the ...
1
vote
1answer
160 views

Scoring predictions of an ordinal variable

I read about using scoring rules to evaluate the performance of predictive models. In the Wikipedia article about the Brier score, it is stated: The Brier score is appropriate for binary and ...
1
vote
0answers
47 views

Correcting a DNA scoring algorithm for scanned sequence length (consensus binding site)

In this paper, the authors present a scoring algorithm for potential transcription factor binding sites, based on the position-specific probability matrix (PSPM) for that particular transcription ...
1
vote
1answer
2k views

Logistic regression: Fisher's scoring iterations do not match the selected iterations in glm

it happened to me that in a logistic regression in R with glm the Fisher scoring iterations in the output are less than the iterations selected with the argument ...
0
votes
0answers
56 views

How to score the options in a questionnaire and develop a scoring range in a risk profiler questionnaire?

I have developed a risk profiler questionnaire but don't know how to score the options and make a scoring range. Once the range is developed, I could categorize the client as conservative, moderate ...
2
votes
0answers
66 views

Creating an index that measures how accurate economsits are at predicting the US economy

I am interested in building an index that tracks how accurate economists are at predicting several US economic statistics: US Jobless claims number: comes out weekly, ranges from 100k to 400k, ...
1
vote
0answers
378 views

How to create a single rank / score number from a multiple set of variables that define the quality of something

I have several variables that describe the quality of a trading relationship between a client and financial institutions (FIs) the client deals with. The variables are populated with observed data ...
4
votes
2answers
468 views

Good books/papers on credit scoring

I'm looking for recomendations of books on credit scoring. I'm interested in all aspects of this problem, but mostly in: 1) Good features. How to build them? Which have been proved to be good? 2) ...
3
votes
1answer
222 views

Validation of a scale for a different population (CFA)

I'm currently adapting a coping questionnaire which has been widely used in sports to be used with a different population (musicians), so I need to address its reliability and validity with this new ...
1
vote
2answers
306 views

How can I estimate a “confidence” in a score / rating

I'm sure there's a more technical term for what I'm looking for, but I don't know what it is (it's not confidence intervals, I think) I'm trying to make a system where people get ratings (from 1 to 5 ...
5
votes
1answer
214 views

How can I devise a scoring system for a competition that is more fair than straight percentages?

I am trying to come up with a method for deciding the winner from among eight student groups competing for a prize. The raw data and corresponding percentages measure participation per group in a ...
1
vote
2answers
389 views

Comparing player percentages effectiveness with different sample sizes

I have a spreadsheet of player data. One of the categories of player data is pass percentage. Pass percentage is calculated as follows: AccuratePasses/TotalPasses ...
2
votes
1answer
179 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
0
votes
0answers
599 views

What helps (justification) to decide cut score/cutoff points for 5-point likert scale?

I am wondering what are the factors involved in deciding the cutoff value / cut score for 5-point likert scale from: ...
1
vote
1answer
298 views

Confidence interval scoring

Question migrated form http://stackoverflow.com/questions/10019576/confidence-interval-scoring-with-programming-languages#comment12812602_10019576 I used a CI scoring algorithm from ...
2
votes
2answers
171 views

Detecting patterns of cheating and collusion in competition

I'm looking for ideas on how to discover cheating in sports that have judges awarding points. I can't name the sport since I participate in it at the moment. It's been known for a long time that this ...
2
votes
1answer
210 views

Scoring new observations after cross-validation

I have some doubts about cross validation and scoring a new set of observations. Let's say I want to predict $y=b_0 + b_1x_1$ and have built a 10-fold cross-validation data set, run a regression ...
1
vote
1answer
187 views

What is the rationale for using scoring methods after logistic regression?

I generally come across scoring in logistic modeling (also other methods like discriminant analysis, etc.). I want to know what is the use of scoring of individuals after running a logistic ...
2
votes
1answer
71 views

Re-estimate classification model with biased data

Let's say that I score (with, for example, a logit model) a group of 10,000 customers according to their potential of buying a product, and that I decide to contact the top 1,000 with a special offer. ...
1
vote
0answers
270 views

How to combine unbound variables with very different frequency distributions?

I want to combine three unbound variables. Each variable is the score provided by three different algorithms. Each algorithm predicts the likelihood (score) that a specific interaction between two ...