Questions tagged [modeling]
This tag describes the process of creating a statistical or machine learning model. Always add a more specific tag.
1,867
questions
0
votes
0answers
2 views
Aggregating/Summing feature importance for categorical fields in xgboost
When I use a standard GBM package for categorical features, I receive importance metrics at that field level.
For example:
State: 0.75
However, when using XGB the importance is split by each of ...
0
votes
0answers
14 views
Fit linear model between two random variables based on their samples
I have two sets independently drawn from two different distributions.
How can I fit a linear model between the underlying random variables?
More precisely:
S1 is a sample drawn from distribution D1 ...
1
vote
0answers
10 views
GARCH(1,1) Variance Targeting
I am implementing a t-Student GARCH(1,1) model using the variance targeting approach. In particular, let's say I have a series of daily log returns ${y_t}$ with $t=1,2,...,262$. My model can be ...
2
votes
0answers
23 views
Weights of Evidence Objections
Weights of Evidence (WOE) is a method used to transform predictor variables in logistic regression. I have not been able to find many objections to WOE as a means of variable transformation for ...
0
votes
0answers
7 views
Concomitant Variables in Finite mixture Models
On what basis does one decide whether a variable is to be taken as a concomitant variable in a finite mixture model (using flexmix in R)? With experimental data this seems easy to answer, but what ...
0
votes
0answers
15 views
Can we use the whole data set to create a model and then cross validate?
Note: I am an amateur data scientist.
I have a weekly level revenue, sales and ad investment data set for 53 weeks. Here revenue is the dependent variable and various sales and ad investment are the ...
0
votes
0answers
31 views
How to write a logit regression in a journal format?
I was inspired by the following question:
How to write a logit and probit regression equation?
To ask for advice on how to write a logit regression equation in a suitable format for an academic ...
0
votes
0answers
8 views
Define optimal Data Volume for Model Development Population?
I would like to ask and learn some approaches, analyzes about representativeness of modeling data. I want to know what is the optimal date range for model development population?
I want to ask that ...
0
votes
0answers
14 views
How to compute prior and likelihood? [closed]
I have some data like distance vs. some magnetic value. I want to fit a model to the data, applying the Bayesian decision. The model parameter is consisting of 6 variables. I am not getting how to ...
0
votes
0answers
19 views
Propensity modeling
I am trying to model for understanding the probability of a customer buying a product based on his/her recent interactions with the system. For a customer who has made a purchase it is very easy to ...
2
votes
0answers
22 views
Testing nonlinearity given linear fit
After fitting a multiple linear regression model, how do you test that the relationship between the response and the predictions is actually linear? For example, if the true relationship is
...
0
votes
0answers
23 views
What are some statistical tests to perform from 8 samples (quantitative data) collected from 4 devices?
I need advice on the best statistical analysis to perform for data I collected from 4 devices. Device A and B are of the same height and are taller than device C and D which are of the same height as ...
0
votes
2answers
46 views
poisson vs ordinal regression
I have an outcome variable in my dataset that follows a Poisson distribution
Y Frequency
0 52121
1 2831
2 34
0 - None, 1 - Moderate, 2- Severe : ...
-1
votes
0answers
35 views
Why is the sum of proportions in susceptible-infected-removed (SIR) models greater than 100?
I would like to test susceptible-infected-removed (SIR) models.
I taken the core code from the Kolaczyk and Csárdi book (Section 8.5, pp. 157-158).
...
0
votes
0answers
12 views
Creating better calibrated models
I am trying to create a model that predicts an outcome for a population cohort. The model is assessed using AUROC for discrimination and calibration plot for calibration. When using decision tree I ...
0
votes
1answer
24 views
multivariate time series analysis with interactions among multiple dependent variables
I have been playing with time series data and using models like facebook prophet which uses time series decomposition and models the signal using trend, seasonalities and holidays.
The stuff I found ...
0
votes
1answer
21 views
excluding important predictors in a model
My colleagues want to test the relationship between a health outcome and demographic variables (race, gender, age, etc.) but don't want include health status (assume this is a critically important ...
2
votes
1answer
22 views
Using predictors observed at different times in regression
I want to predict vegetable yields for each field using samples of the vegetables.
Some fields have multiple samples taken over the season and some only have one sample. Sample data consists of ...
2
votes
1answer
32 views
What are some of the most correct/accepted ways to tune and compare different models in an academic context?
Those days, I have been reviewing different academic papers which mainly compare the performance of different machine learning methods on a particular problem. And I was surprised by the variety of ...
0
votes
0answers
13 views
Regression with a growing year over year dataset
I created a hierarchical GAM to model final event sales as a function of sales to date, days until event, teams that are playing, and event month with a grouping at the home team level. This yielded ...
1
vote
0answers
28 views
What are the differences between probabilistic models, stochastic models, and statistical models? [closed]
What are the differences between probabilistic models, stochastic models, and statistical models?
Do statistical models deal with data sets, and model them mathematically to capture the summary ...
0
votes
0answers
23 views
Best approach to an event detection classifier model
Probably my title is not as precise as I'd like it to, but bear with me, the problem is quite straightforward: I have daily time-series sales and stock data, from January to May, both in 2018 and 2019....
0
votes
0answers
12 views
Training an LSTM model for text summarization with different type of different domain data for training vs test
If I want to use an LSTM for doing text summarization but the labeled data I have (as in the summarizes which represent the labels) is from a different domain (Amazon reviews) but I have over 500K of ...
3
votes
0answers
31 views
Check whether data forms a curve or not [closed]
I have a set of lists and need to determine if the data of every list forms a curve. The picture below is an example of how the data looks plotted. Humanly it is easy to identify which ones are the ...
1
vote
3answers
34 views
Interaction terms in regression when variables can be negative
Suppose I have a regression
$y = b_0 + b_1 x_1 + b_2 x_2 $
where both $x_1$ and $x_2$ have a range from $-\infty$ to $\infty$ and have been centered.
The correlation between $x_1$ and $x_2$ is $0.3$...
0
votes
1answer
18 views
Conceptual interpretations of predictive versus causal modeling
I'm a relatively non-technical clinician. And I recently came across an excellent paper paper (Schooling & Jones, 2018).The authors essentially implore biomedical researchers to use non-ambiguous ...
0
votes
1answer
26 views
Do I add both the new variable and the original variable to my log binomial model?
I'm calculating interaction indices for some variables in my model. I have an interaction between two variables (age (continuous) and hypertension (binary)), and I read in a similar publication that ...
2
votes
0answers
13 views
What model to use to evaluate whether the “variability” of one variable is determined by the “variability” of other variables?
I have observations taken for four subjects at a number of points/locations. The data is formatted similarly to as below:
I am not directly interested in the impact that a predictor has on the ...
2
votes
2answers
59 views
How do I use a beta distribution to model this bimodal data?
I am trying to model data that looks like this:
The data reflects frequencies of items at 0, 25, 50, 75, and 100. The majority of the time, values are concentrated at the 0, and 100 points. I believe ...
0
votes
0answers
7 views
Probability model vs statistical model vs stochastic model
I understand that a statistical model is a model which accounts for the uncertainty in the model. Eg. Demand price equation: $demand_{i}= a+b price_{i}+ u_{i}$ where $u_{i}$ refers to the residual ...
1
vote
0answers
41 views
Survival analysis: why use a continuous probability distribution?
Estimating the time to an event (survival analysis) is typically modelled in continuous time, assuming the time to event is exponentially distributed (or gamma, Weibull, log-normal).
However, as ...
0
votes
0answers
14 views
How a dataset can be modeled as Stochastic Process
I have a question about time series and stochastic processes representation in python
Recently, I read this web page for modeling the temperature dataset and it considers a time series as a ...
1
vote
2answers
46 views
addressing the effect of the independent variable on the dependent variable for 2 different types of individuals
I am estimating the effect of a continous treatment X (that goes from 0 to 1) on a dependent variable y (data is taken through an experiment).
I have around 250 Individuals in my dataset that can be ...
0
votes
1answer
17 views
feature engineering - dealing with conditional numeric variable
Let us say I have a dataset with 2 columns X1 and X2. X1 is dichotomous and has 2 level yes and no. X2 is numeric and is only completed when X1 = yes. Is it possible to keep the numeric column X2 as ...
0
votes
0answers
25 views
Days Between Visits Model
I have a dataset from a digital service on a user-level and want to model the decrease in visits and usage over time. I want to explain the decrease (or increase) with the activities a customer ...
0
votes
1answer
15 views
K-fold partitioning for dataset consisting of discrete, sampled sub-datasets
I can observe full system dynamics from some deterministic start to some deterministic end, in order to collect a sub-dataset $\boldsymbol{X}_i$ with $m$ examples, where the distribution of these $m$ ...
0
votes
0answers
4 views
Help on whole model specification and guide on procedures
Could you guide me what I should do in my research in terms of econometrics.
I want to emphasize the causality in energy savings by buildings and various factors (like climate etc.). I don't have a ...
15
votes
7answers
2k views
What kind of curve (or model) should I fit to my percentage data?
I am trying to create a figure which shows the relationship between viral copies and genome coverage (GCC). This is what my data looks like:
At first, I just plotted a linear regression but my ...
0
votes
0answers
33 views
Logistic regression vs segmentized logistic regression
This might be a very rookie question. I need help interpreting the results of my logistic regression.
Assume I have the following model: y ~ categorical + numerical...
0
votes
0answers
30 views
Specifying interactions as predictor variables in a GLMM R formula - correct method?
I am modelling insect count data as a GLMM of distance along a transect (the length of a growing tunnel (m)), temperature along the tunnel and external daily max temperature, ie in R-
Insect_count ~ ...
0
votes
0answers
11 views
Regressing age-specific outcome for individuals born in successive years from biennial longitudinal survey
My question is about a regression model. I am using longitudinal survey data on individuals born in successive years that is collected every two years (in even years) from interviews. I want to ...
0
votes
1answer
40 views
Modeling non-linear (short) time series and cross-validate them
beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense.
I have many multivariante short time series, between ...
2
votes
1answer
45 views
Waiting time distribution parameters given expected mean
I have a set of healthcare providers serving patients. In a given amount of time, a specific provider can see only a certain amount of patients, depending on the medical procedure and other variables.
...
9
votes
1answer
481 views
What is the likelihood for this process?
A patient is admitted to the hospital. Their length of stay depends on 2 things: The severity of their injury, and how much their insurance is willing to pay to keep them in the hospital. Some ...
1
vote
0answers
22 views
How can I calculate the exponent (parameter or tail index) of the Pareto distribution associated to the following paperclips experiment?
The experiment is the following:
take a pool of paperclips for which in each round we follow these steps:
1) choose two paperclips randomly
2) if they are not linked with each other then link them ...
1
vote
1answer
18 views
Change in X and Change in Y in Pearson Correlation
Suppose I have my x variable calculated as Hormone2-Hormone1 where Hormone2 is at a later time point and Hormone1 is earlier - this is true for all subjects and the time points are equally spaced for ...
0
votes
0answers
32 views
Estimate total effect of treatment when continuous treatment variable and its dichotomization show contradicting impacts
I am trying to estimate the total effect of a leak of a harmful gas on my dependent variable. While I do not know the actual exposure, I know the distance of a household to the origin of the leak in ...
2
votes
1answer
25 views
Comparing model performance. What model should I run for this? Confused between Randomized block or Repeated measures
I am trying to analyze some results I have generated from multiple different models. I have 21 models which are all used on the same 9 sets (years) of data. The response variable is the result of the ...
0
votes
0answers
17 views
How can I maximize revenue given a distribution of order totals?
I need help with a statistical modeling problem in R. I want to determine the optimal shipping policy (i.e. free shipping minimum and standard ship rate) to maximize revenue for an e-commerce site. I’...
1
vote
0answers
29 views
Include main effect when specifying varying coefficient models with GAMM?
I am trying to model an interaction between two continuous variables in the GAMM framework using gamm in R. For simplicity's sake, I am ignoring the repeated ...
