The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
14 views

Using likelihood to compare model accuracy on two different data sets

I have a model that produces conditional probabilities, $p(Y|X)$, where $Y$ is either 0 or 1, and $X$ is just some random variable. I have two different data sets $Z_1, Z_2$ consisting of pairs of ...
0
votes
0answers
18 views

Compute probability from distance-score

I compute Euclidian distances between a point I want to analyze and a set of points I have. I want to sort my points by descreasing "similarity". I used to compute a "score" by inverting the distance ...
0
votes
0answers
9 views

Proper Method to Add weights to variables

Given a dataset with different fields of criteria, all coded as 1 or 0, and also with a target which can take a value of 1 or 0, how can I create weights into the fields. In a banking example, let say ...
0
votes
0answers
16 views

Developing a scoring method for groups of devices

I am hoping some of you smart statisticians can help me with a problem I am trying to solve. I am trying to build a model that will create a “Score” for sets or groups of devices that exist within ...
0
votes
1answer
12 views

How to assign scores

I have 3 categories of responses (A, B and C)for a survey question. I want to give a score of 5 (to A), 10 (to B) and 15 (to C) respectively according to the importance that i attach to them. What is ...
0
votes
0answers
11 views

Find performance of each group without penalise small groups

I have a table with the following data: UserId, TotalPoints, GroupId I try to find a formula to calculate the performance of each group in the above table. I ...
0
votes
0answers
11 views

Budget scoring algorithm

I'm working on a free budgeting app that needs to calculate a few summary scores on the data entered. This is for personal budgeting so no need for enterprise size complexity. I have not done any ...
1
vote
0answers
26 views

How to calculate influence of variables at ROW LEVEL?

There are several algorithms which give relative importance of variables at OVERALL Model level. But the most influencing variable might not be the reason why a particular row might get higher or ...
0
votes
0answers
29 views

How to scale two models build for two different parts of the same data set the way I can bind the predictions and use them as one final model?

I have a big data set where two models have been built. Every model is used to make predictions of some part of the data, but I want to measure how the two models combined are working in the whole ...
0
votes
0answers
18 views

Using R to calibrate model probabilities back to the full US population after down sampling

I am using the survey package and my model is: ...
0
votes
0answers
13 views

Score combination: “additive” averaging?

I'm computing multiple scores for a given person, on a range from 0 to 100. I'd like to compute an overall score per person, with the catch that both the total number of scores and the total amount ...
0
votes
0answers
21 views

Logistic regression - 2 models don't improve KS instead of one

I am trying to order by probability of buying insurance a group of clients. Since there are new clients, the variables that I'm using might be ruined if I use them in the training set. First I ...
0
votes
1answer
37 views

Judgement score regularization problem

Consider the scenario where M performances (eg. singing contest) are being judged by N judges. Each judge awards a score S(m,n) to each performance on a scale of one to one-hundred. The problem ...
0
votes
0answers
20 views

Does this type of diagram showing conditional mean have a name?

I'm trying to show the skill of a prediction $\hat{Y}$ relative to a true value $Y$ beyond simple statistics like MSE or $r^2$. I usually like looking at the scatter plot of $\hat{Y}$ vs $Y$ but in ...
0
votes
1answer
898 views

Probability of a team scoring a goal?

I am trying to work out the probability of teams scoring a goal in a football match. Obviously there are many factors to consider, and the probability is impossible to predict correctly as there are ...
0
votes
0answers
28 views

taking decisions: how to combine scores from different systems?

I have several completely different software components that are producing scores for certain candidate solutions. Now, setting aside the technical details of what is the problem that I want to solve ...
0
votes
0answers
177 views

Why the Brier Score's better when probabilities are estimated through PAVA instead of Platt Scaling?

I've been studying (and applying) SVMs for some time now, mostly through kernlab in R. ...
0
votes
3answers
112 views

Improving quality of logistic regression estimation

I'm working on a credit scoring model (logistic regression), and I have divided my dataset (5082 obs with 580 negatives) in two samples: 75% training set and 25% test set. The result of the estimation ...
0
votes
0answers
61 views

How to build kpi on employee<>customer<>sales

For further analysis i want to build a KPI which describes the "critically" of a employee<>customer relationship. To make it clear, some example data: Employee - Customer - ...
0
votes
1answer
59 views

What is the prediction equation for penalized logistic regression?

I have used penalized logistic regression (R package logistf) to predict probability of a rare event. 0.12% is the event rate i.e., only 35 occurrence of event in ...
1
vote
1answer
25 views

Is the “angle=arccos(spherical)” scoring rule strictly proper?

The Spherical scoring rule is known to be strictly proper. However, it is not very intuitive. It's arccos, however, is the angle between the prediction ...
6
votes
1answer
165 views

Why does Lucene IDF have a seemingly additional +1?

From the Lucene docs $\text{IDF} = 1 + \log\left(\frac{\text{numDocs}}{\text{docFreq}+1}\right)$ In other references (i.e. wikipedia), IDF is typically calculated as ...
0
votes
1answer
43 views

Is there a distance algorithm similar to Jaccard distance that handles scalar data?

we have the characteristics and (scalar) values of those characteristics for three (or more) people: ...
0
votes
0answers
23 views

scoring/predicting for new observations

I have two data sets of variables where one of them - the new observations - has no dependent variable. The data set without a dependent variable has around 20 times the number of records. ...
0
votes
0answers
72 views

Regression: Predicting values over orders of magnitudes - what metric(s) to use?

What is a good metric to use for predicting values over several orders of magnitude? I can use R^2 but other measures like mean absolute error or mean squared error are pretty meaningless. In my case ...
4
votes
1answer
8k views

Stepwise Model Selection in Logistic Regression in R

I'm implementing a logistic regression model in R and I have 80 variables to chose from. I need to automatize the process of variable selection of the model so I'm using the step function. I've no ...
1
vote
1answer
65 views

Cut Score Determination in IRT

Cut score or cut point is a minimum achievement or ability that examinees should get in order to get a 'Pass'. In CTT, MCQ scoring in the form of percent-correct scores are applied. Let say we ...
1
vote
1answer
73 views

optimal down payment estimation in credit scoring

Knowing I can estimate the risk of default, via logistic regression, of a consumer on a small loan... what would be the best way to estimate the optimal down-payment amount to ask for in order to ...
6
votes
1answer
386 views

Justifying and choosing a proper scoring rule

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
1
vote
0answers
830 views

Bootstrapping in SAS - PROC LOGISTIC - Next steps ? how to score / perform diagnostics?

My question is as follows. I am referencing the following paper by David Cassell - wherein David talks about bootstrapping techniques in SAS using PROC SURVEYSELECT (many thanks to David - truly a ...
1
vote
1answer
460 views

Survfit function in R to score a new dataset

I have built a cox proportional hazards model in the R survival package. I want to score new data set using this model. I thought the survfit function would doing this using survfit(original model, ...
2
votes
1answer
160 views

Using proper scoring rule to determine class membership from logistic regression

I am using logistic regression to predict likelihood of an event occurring. Ultimately, these probabilities are put into a production environment, where we focus as much as possible on hitting our ...
1
vote
0answers
50 views

Proper scoring rules for observations with different supports

Suppose to have a bivariate variable $z_t=(x_t, y_t)$ indexed by $t=1,2, ..., T$. Suppose now that the two components have different support, i.e. in my specific problem $x_t \in \mathcal{S}$, where ...
1
vote
1answer
877 views

What data mining/machine learning approach to use for a scoring model?

Suppose I have a large data set with lots of features(attributes). And I'm tasked to build some kind of scoring model to rank certain objects with all these features. How do I go about doing this? ...
11
votes
2answers
278 views

Intelligence Squared Scoring and Winner Determination

There is an NPR podcast called Intelligence Squared. Each episode is a broadcasting of a live debate on some contentious statement such as "The 2nd amendment is no longer relevant" or "Affirmative ...
1
vote
1answer
65 views

Algorithm for scoring co-varying traits

I am sure this has been done, but I can't find quite the right approach. EDIT: Trying to explain better. The rows of colored boxes below are columns of molecular sequence data -- positions in a ...
4
votes
1answer
109 views

Relation between Scoring rule and Loss function in Parameter estimation and model selection?

Initially, I had only heard of MLE and use it for almost everything, e.g. point estimate and model selection (with some penalty). Then, MSE appeared, which seems to play the same role as MLE does. I ...
0
votes
0answers
44 views

Creating a model to interpret numerical scores

Good morning/afternoon everyone, first of all thanks to all of you for the valuable insights provided. I will be oulining here my current challenge, trying to provide as much detail as possible. ...
1
vote
0answers
23 views

Is there a reason to score on subsets of your data?

If you're training a model using cross validation on one set of data, and then you're scoring that model on a separate set of data, is it advised to score, say, 1000 times on subsets of your data? ...
1
vote
0answers
541 views

Random forest “certainty / likelihood score” - how to score records in RF mode in R?

My question is similar to this link Creating a "certainty score" from the votes in random forests? I am trying to build a random forest for a binary response (1 & 0). Let's say we have ...
0
votes
2answers
61 views

How to get an “averaged” score?

Assume a game wherein a character's "power" is made up of several factors, like speed, weight, body build, etc. And let's say that each of these factors were scored 0, 1, 2 wherein 0 means average, 1 ...
2
votes
1answer
388 views

Cluster many thousands observations (mixed variable types). Cluster subsample and then classify the rest observations?

I'm trying to run a cluster analysis on a large dataset (70k+ observations to cluster) with mixed variables (numeric, ordinal, binary and nominal). I don't think I can create the distance matrix using ...
3
votes
1answer
87 views

Deploying survival model for production

I've recently built a survival model with time-varying covariates (assuming equal time periods) using R and I am now in the process of putting this into Oracle for production using just the ...
1
vote
1answer
237 views

Scoring predictions of an ordinal variable

I read about using scoring rules to evaluate the performance of predictive models. In the Wikipedia article about the Brier score, it is stated: The Brier score is appropriate for binary and ...
1
vote
0answers
48 views

Correcting a DNA scoring algorithm for scanned sequence length (consensus binding site)

In this paper, the authors present a scoring algorithm for potential transcription factor binding sites, based on the position-specific probability matrix (PSPM) for that particular transcription ...
1
vote
1answer
3k views

Logistic regression: Fisher's scoring iterations do not match the selected iterations in glm

it happened to me that in a logistic regression in R with glm the Fisher scoring iterations in the output are less than the iterations selected with the argument ...
0
votes
0answers
63 views

How to score the options in a questionnaire and develop a scoring range in a risk profiler questionnaire?

I have developed a risk profiler questionnaire but don't know how to score the options and make a scoring range. Once the range is developed, I could categorize the client as conservative, moderate ...
2
votes
0answers
74 views

Creating an index that measures how accurate economsits are at predicting the US economy

I am interested in building an index that tracks how accurate economists are at predicting several US economic statistics: US Jobless claims number: comes out weekly, ranges from 100k to 400k, ...
1
vote
0answers
679 views

How to create a single rank / score number from a multiple set of variables that define the quality of something

I have several variables that describe the quality of a trading relationship between a client and financial institutions (FIs) the client deals with. The variables are populated with observed data ...
5
votes
2answers
873 views

Good books/papers on credit scoring

I'm looking for recomendations of books on credit scoring. I'm interested in all aspects of this problem, but mostly in: 1) Good features. How to build them? Which have been proved to be good? 2) ...