Questions tagged [monitoring]
The monitoring tag has no usage guidance.
15 questions
1
vote
0
answers
9
views
Risk score developement from admin health data: when the test for the risk factor and outcomes are confounded by indication
I have health records of immunodepressed patients who may have event histories like
[high risk demographics] -> [low lymfocyte count] -> [high viral load] -> [clinical events]
From those data ...
0
votes
0
answers
32
views
Daily monitoring of churn prediction model
I've written and trained a churn model that is scheduled to run every day and make new predictions for the probability of each customer to churn within coming 365 days, from the day the scoring is ...
- 363
1
vote
0
answers
7
views
Which strategies,framework and applications tools can be implement to automatically monitor the health of the machine learning model?
ML Models when deployed in the production environment, model degradation can arise where their output will change if the relationship between the incoming serving data and the predicted target drift ...
- 173
2
votes
2
answers
69
views
Analyzing data from a non-randomized sampling design (ecological monitoring)
I have 2 questions about analyzing data that was not randomly sampled from a population.
I work with "ecological monitoring" data that involves repeatedly taking measurements from the same ...
- 2,061
2
votes
0
answers
23
views
Detecting behaviour change in a recommender system
Suppose I want to track a recommender system's live performance. The task isn't exactly to detect outliers, but to detect if the system started behaving differently, looking at the output only - an ...
- 33
5
votes
1
answer
147
views
What does it mean if the distribution of the scores given by a classifier change over time?
I´m currently learning about how to determine if your model keeps performing well or has degraded (particurlarly, for classification problems). My question is, what kind of info can I derivate if I ...
- 269
3
votes
0
answers
41
views
When and how to re-evaluate deployed models
There are a lot questions about how to train a model (family) or how to tune hyperparmeter. But there are surprisingly few question about how to monitor or evaluate a model already in production (...
- 3,661
1
vote
0
answers
21
views
Which control chart can use for monitoring of covariance matrix of cofficient?
I have a series of panel data, which includes p cross section and t time. I estimated the covariance matrix of coefficients for periods for periods.
Now my data includes the individual period and the ...
- 11
1
vote
0
answers
30
views
Estimating water quality statistics using multiple sites
I have a typical water quality data set, containing a number of sites sampled across a couple of years, generally on a monthly timescale.
I want to estimate the 80th %ile for setting water quality ...
- 11
2
votes
0
answers
597
views
How to detect distribution changes within population?
I have a time series data of populations, e.g. for each time point, I have the blood pressure of 1 million of rats. How can I monitor that the distribution of this population drift over time?
For ...
- 31
3
votes
0
answers
228
views
Howto derive statistical upper limit in case of zero observation of poisson process? [duplicate]
I am writing software, that keeps track of hourly rates of certain incidents in historic data. I find that the generating process of these incidents is acceptably well described by a poisson process.
...
- 627
1
vote
1
answer
410
views
How to monitor and alert on sparse event data?
To give you an idea of what I'm dealing with, I calculated the average time between events for each of our customers. The lowest is 24 seconds and the highest is 689,014 seconds. Yes, 7 days.
I'm ...
- 131
4
votes
1
answer
277
views
Testing for a drop in bookings
We're developing real-time alerts for fine-grained (every 5 minutes) time series bookings data, and I'm looking for the best approach to doing this. Idea is that if over the past 10–15 minutes (say) ...
user11284
6
votes
1
answer
202
views
Dependent variable selection for loglinear segmented regression in time-series analysis of rare events
My office is going to implement a bundle of infection control measures in hospital and see if it can effectively reduce the infection rate of some pathogen. The unit of measurement will be "case per ...
- 3,249
6
votes
2
answers
233
views
How to tell if something happened in a data set which monitors a value over time
I have a data set where a series of measurements are being taken each week. In general the data set shows a +/- 1mm change each week with a mean measurement staying at about 0mm. In plotting the data ...
- 607