Questions tagged [modeling]
This tag describes the process of creating a statistical or machine learning model. Always add a more specific tag.
2,545
questions
0
votes
0
answers
26
views
PROC MIXED HELP [closed]
I am just starting out with PROC MIXED and unable to find help at my university, so hoping someone here can help!
I am setting up a model where my outcome (weight) is measured 3 times (baseline, 12m, ...
- 11
0
votes
0
answers
21
views
gamm4 output size is too large (~5GB). How to decrease? [closed]
I am running a Generalized Additive Mixed Model with the R package, gamm4. Each model output includes a mer object and a gam object. I need to compare 26 model structures based on a combination of ...
0
votes
0
answers
14
views
Mixed Models two continuous time-points
I'm using a mixed model to do a hypothesis test of an intervention effect (group: 0=control, 1=intervention) in an RCT with two time-points: Baseline plus follow-up 6 months after baseline. In ...
- 21
0
votes
0
answers
12
views
Compare and aggregate indicators at different geographic scale
I encountered a problem in developing an aggregation model of several indicators. Almost all the indicators are at a small geographical scale (admin2 / Provinces), but one is at larger geographical ...
0
votes
1
answer
38
views
Practical usage of the bias variance tradeoff
I understand the bias-variance tradeoff. But, I have never come across a scenario where that has changed anything in the modelling process. Is there any practical scenario that you have encountered ...
- 2,263
2
votes
0
answers
15
views
Reporting AIC values for model fits with multiple runs - report the min or the average?
I have a model that I fit with NLL minimization.
I fit that model n times with random starting values, to try to avoid local minima.
When reporting the results, it seems reasonable to me to report the ...
1
vote
1
answer
69
views
Using feols() vs plm() vs lm() in panel regressions in R
I am using panel data at the district level. My outcome variable is the share of employed individuals in a given district. I am regressing this variable on a binary treatment dummy called "treat&...
- 177
0
votes
0
answers
29
views
Quantifying fit of a non-linear model
I have developed a non-linear model that utilizes 0-fitting parameters (all my inputs can be/are determined experimentally). As of now I have performed propagation of error and a GSA using Sobol ...
- 1
0
votes
0
answers
15
views
Simulation data vs measure data
I am comparing climate simulation data (in reanalysis mode) to in situ measured data (from a weather station).
For now, the main variable I study is snow height (in cm).
Here is an exert of my data (...
1
vote
0
answers
5
views
Creating a VAR model - data with seasonalities
I am in process of creating a forecast using VAR model for pricing of a certain commodity.
Some of my variables (such as price itself, as well as inflation, and taxation) don't have any seasonalities. ...
- 11
0
votes
0
answers
18
views
How to estimate the effect of time series on a variable sampled irregularly and much less frequently for multiple subjects
I’ve been struggling for several days to find the proper statistical analysis tools for my problem, and I’m hoping for some valuable tips and insight from the internet. I’ve read up on ARIMA(X), ...
0
votes
0
answers
29
views
Can LASSO still perform regularization on summarized data?
Currently, we are trying to predict future revenue from existing users. We use the revenue collected after 14 days of membership to predict 3 year membership. We train the model and make predictions ...
- 32.2k
0
votes
0
answers
9
views
Prognosis modelling data set
I want to externally validate and update a previously developed and published model (Model A) in my dataset. In the case of poor performance, I would like to develop a new model for the same outcome (...
- 23
0
votes
1
answer
40
views
Predicted cloud of points is tilted compared to the diagonal [duplicate]
I built a model with 2 variables, i found that the predicted values for the test set did not align with their real values, so i predicted instead with the data which were used to train the model. I ...
1
vote
0
answers
66
views
How to create representative training, validation, and test sets when working with time series data?
In my application, I am working with a relatively long time series of daily market index percentage returns (many years) and am trying to model the dependence structure of the returns from a pure time ...
- 185
2
votes
0
answers
27
views
Functional Principal Component Analysis - Explaining Functional Principal Component Scores
I was wondering if someone can help with explaining Functional Principal Component Scores?
I am working with a dataset which reflects participants in a weight loss management trial (longitudinal data)....
3
votes
2
answers
85
views
Why do we seek to find a model that approximates all data instead of finding a function that fits all data?
I tried to learn the principle idea of regressions. As I understand, the aim is to find a model that represents the relation between the x numbers and the y numbers, so that we can understand the ...
- 33
0
votes
0
answers
24
views
What's the Issue with Modeling Admissions as Multinomial?
I am reading Chapter 4 (Testing and Confidence Regions: Basic Theory) of Mathematical Statistics by Bickel and Doksum. In Example 4.1.1, the authors use the study of sex bias in graduate admissions at ...
- 123
0
votes
0
answers
33
views
Is there a statistical model for the problem I want to solve?
I have two time series features, $A$ and $B$: both returns. When $A$ is at 0 (as it is for the vast majority of the dates), you cannot say anything about $B$; it's just noise. However, when $A$ ...
- 197
1
vote
0
answers
14
views
Develop a global model for multiple objects
First of all, i don't know if "global model" is the right name for this.
The goal is to develop one model to predict the solar energy of PV-Systems in an energy management system for the ...
- 11
1
vote
1
answer
20
views
Geometric meaning of cross product of a design matrix and model coefficients
I am trying to understand more about the geometry of linear modeling. Take for example an experiment with one categorical predictor having two levels and a numerical response. If two data points are ...
- 139
1
vote
1
answer
9
views
Difficulties to decide on model formulation and to identify dependent and independent variables
Imagine a disease whose exact mechanisms are unknown and can lead to different phenotypes. Although there likely is a continuum between the different phenotypes, these are categorized (as ordinal ...
- 344
1
vote
0
answers
20
views
Gaussian white noise model in application
I am interested in applications (to data) of non-parametric statistics, and my question concerned the Gaussian white noise model defined by,
$$
X_{t_1, \ldots, t_d}=f\left(t_1, \ldots, t_d\right) d ...
- 11
1
vote
0
answers
31
views
Using both Random Forest and RFE for Feature Selection and Dimensionality Reduction [closed]
I am using random forest classifier currently to do feature selection for a balanced dataset of about 18K rows and 7050 features. I recognize this is a lot of features.
I am thinking of using random ...
- 11
0
votes
0
answers
13
views
What kind of model do I use to calculate the probability of an observation falling into each of several categories?
First off, I don't have a heavy data science background, so if I have the completely wrong approach for this, a pointer in the right direction would be appreciated.
I am trying to build a series of ...
- 141
0
votes
0
answers
18
views
Defining parameters so that they obey multiple constraints
I'd like to define parameters $\beta_i$ for $i=1,\ldots,I$ for a problem so that they automatically obey some constraints. The constraints are:
$\sum_{i=1,\ldots,I} w_i \beta_i = c_1$ and
$\sum_{i=1,\...
- 29.3k
0
votes
0
answers
29
views
Time series: how to model the impact of exogenous variable?
Disclaimer: this is a more theoretical question than a practical one.
I am trying to predict the revenues of a company (so I can't post the data) using the past 4 years data (monthly) and exogenous ...
- 101
0
votes
0
answers
37
views
Dirichlet Regression output and using the calculated coefficients in regression model
I am very new to Dirichlet Regression and trying to make sense of the output and the regression coefficients. I am doing a biomass study and have tested the following variables (DBHH, DBH + H, DBH and ...
- 1
0
votes
0
answers
25
views
Non-Linear data fitting for kinetic data
I am trying to fit a two_site model which I derived, (basically, rate as a function of pressures of reactants and products) to my experimental data. Just want to check if the logic below is fine, ...
0
votes
0
answers
25
views
Can confounders be controlled for in an Interrupted time series and when should outcomes be modeled as binary rather than aggregated rates?
I just learned about interrupted time series and have a few questions about them.
Say I have a dataset of individual patients and I want to compare their monthly rates of getting a certain lab test ...
- 51
0
votes
0
answers
20
views
R codes for Bayesian Credible Intervals and p-values for a Additive Bayesian Network model
I am using Additive Bayesian Network (ABN) to model the relationship between cereal crop production and climate variables and want to generate credible intervals and p-values for my model coefficients....
0
votes
0
answers
29
views
Is there a way to estimate dependence between an observed variable and an unobserved variable?
I'm attempting to do some open-ended exploratory analysis/model building on real-valued time series data.
I am explicitly not assuming that all elements of my class of models are linear in the lagged ...
- 185
0
votes
0
answers
24
views
How do i write equation for a lognormal graph if I have 100 points on the graph that clearly have shape of a lognormal graph and the x any are known?
Days 1 to 100 are along the x-axis. The y-axis has the PDF values such that the area under the graph is 1. (x,y) is known for each point. The points line up in a near perfect lognormal distribution ...
- 11
1
vote
0
answers
27
views
Should an offset be removed if the variable doesn't seem to have much influence?
This is about a joint project with a subject matter expert, and I'm the statistician. I will not explain the background and simplify the statistics problem as the background is rather complicated. The ...
- 19.4k
1
vote
0
answers
17
views
PLSR: trait vs spectroscopic data gives very low R2 on plsr model in R
here is the sample data.
I have spectroscopy data as X-variables (from X1 to X80) and corresponding Y variable.
I need to run plsr model in R using "pls" package. There are two sheets. In ...
- 11
0
votes
0
answers
29
views
Quantile regression and prediction estimate
I am working on a personal project and I've been wondering about something.
I'm basically taking some loan data from kaggle, analysing/cleaning it, then trying to apply machine/deep learning models to ...
0
votes
0
answers
13
views
ML (NN) model for a physical process (layer movement in ensilage)
The problem is, I have a several silages with several layers of some substance in them (e.g. coal). Each layer has its own physical/chemical properties (concentration of element, X). Concentration of ...
- 111
0
votes
0
answers
10
views
Statistical models or problems equipped with methods that guarantee optimal or near solutions in finite time
What are some statistical models or problems that have computationally tractable solutions and are guaranteed to be optimal or near optimal? By computationally tractable, I mean finite time and ...
- 2,386
1
vote
0
answers
43
views
Popcorn- what's the best way to compare the fit non-linear models for data that is not very complex?
I popped 7 bags of popcorn in 30 second increments and found the percentage of popped kernels in each time frame.
I manipulated this data with sinusoidal, exponential, and cubic models. There is lots ...
- 11
11
votes
4
answers
2k
views
Methodology for Reconciling "all models are wrong ..." with Pursuit of a "Truer" Model?
When attempting to model various phenomena, in practice, our model will usually be "wrong", but can still provide us with useful insights/predictions.
What methodologies can we follow to ...
- 185
4
votes
2
answers
278
views
Modeling continuous interaction effect with very small sample
I have a very small data set (n = 12) of high level athletes each with a set of measurements representing technique (5 variables) and a set of measurements representing strength (3 variables), and I ...
user389774
0
votes
0
answers
31
views
How to model breakage events for electronic devices using a Weibull distribution?
I got age data (in years) of gas heatings from a random sample of about N=700 participants.
So participants were asked how old their gas heating is in years.
Now I like to model that age distribution (...
- 3
0
votes
0
answers
22
views
Why am I missing the result of a smoothing effect in a GAMM while interpreting results from summary command
I have been using GAMMs to analyze time series data and I have included a smoothing term (hour of day by season) and I can't seem to find the results for the winter season. I have the proper ...
0
votes
0
answers
17
views
How to interpret smoothing effects in the summary output of a generalised additive mixed effect model GAMM
I have been using GAMMs to analyse time series data and I have included a smoothing term (hour of day by season) and I cant seem to find the results for the winter season. I have the proper ...
Kevin Short
0
votes
0
answers
13
views
GARCH fit to the residuals of AR/ARMA mean equation previously fitted
Suppose I have an ARMA (p,q) (let it be ARMA (2,2)) fitted to my original returns series and have the residuals of said ARMA model extracted.
Next, it is my understanding that I need to fit a GARCH ...
- 1
0
votes
1
answer
36
views
Feature importance/model exploration in a large survival model
I've built very-well performing survival model (weibull proportional hazards model for interval censored data, modelled with IcenReg) with many covariates, some ...
- 53
5
votes
1
answer
201
views
Ranking of multiple raters with partially overlapping assessments
Ranking of Raters and Cases
I am looking for a method for ranking a set of raters with partially overlapping binary assessments. I also want to rank the cases according to their difficulty.
Context
I ...
- 59
9
votes
1
answer
643
views
What do I do when a false negative is far more expensive than a false positive?
I'm not sure how to 'responsibly' balance my model to account for this. I could predict a probability and give that to the business ('predict_proba' in SKlearn) but experience in the past has thought ...
- 193
0
votes
0
answers
19
views
Effects of multiple non-independent treatments
I need to run an analysis of efficiency of automated recommendations for call-centre agents on how to handle customer support tickets. It works like that:
A ticket get 0, 1 or more recommendations ...
- 1
2
votes
2
answers
146
views
How to deal with noisy observation in Survival Analysis
I'm new to Survival Analysis. Usually in survival analysis, we want to model the survival function progress w.r.t time. This is normally done through Cox model, or KM-model within a specific time ...
- 165