Questions tagged [modeling]

This tag describes the process of creating a statistical or machine learning model. Always add a more specific tag.

Filter by
Sorted by
Tagged with
0 votes
0 answers
26 views

PROC MIXED HELP [closed]

I am just starting out with PROC MIXED and unable to find help at my university, so hoping someone here can help! I am setting up a model where my outcome (weight) is measured 3 times (baseline, 12m, ...
S P's user avatar
  • 11
0 votes
0 answers
21 views

gamm4 output size is too large (~5GB). How to decrease? [closed]

I am running a Generalized Additive Mixed Model with the R package, gamm4. Each model output includes a mer object and a gam object. I need to compare 26 model structures based on a combination of ...
megsruppUNBC's user avatar
0 votes
0 answers
14 views

Mixed Models two continuous time-points

I'm using a mixed model to do a hypothesis test of an intervention effect (group: 0=control, 1=intervention) in an RCT with two time-points: Baseline plus follow-up 6 months after baseline. In ...
Sebastian's user avatar
0 votes
0 answers
12 views

Compare and aggregate indicators at different geographic scale

I encountered a problem in developing an aggregation model of several indicators. Almost all the indicators are at a small geographical scale (admin2 / Provinces), but one is at larger geographical ...
Dorianeve's user avatar
0 votes
1 answer
38 views

Practical usage of the bias variance tradeoff

I understand the bias-variance tradeoff. But, I have never come across a scenario where that has changed anything in the modelling process. Is there any practical scenario that you have encountered ...
figs_and_nuts's user avatar
2 votes
0 answers
15 views

Reporting AIC values for model fits with multiple runs - report the min or the average?

I have a model that I fit with NLL minimization. I fit that model n times with random starting values, to try to avoid local minima. When reporting the results, it seems reasonable to me to report the ...
Danny Garside's user avatar
1 vote
1 answer
69 views

Using feols() vs plm() vs lm() in panel regressions in R

I am using panel data at the district level. My outcome variable is the share of employed individuals in a given district. I am regressing this variable on a binary treatment dummy called "treat&...
Jerry's user avatar
  • 177
0 votes
0 answers
29 views

Quantifying fit of a non-linear model

I have developed a non-linear model that utilizes 0-fitting parameters (all my inputs can be/are determined experimentally). As of now I have performed propagation of error and a GSA using Sobol ...
Thu21's user avatar
  • 1
0 votes
0 answers
15 views

Simulation data vs measure data

I am comparing climate simulation data (in reanalysis mode) to in situ measured data (from a weather station). For now, the main variable I study is snow height (in cm). Here is an exert of my data (...
Benjamin Imbach's user avatar
1 vote
0 answers
5 views

Creating a VAR model - data with seasonalities

I am in process of creating a forecast using VAR model for pricing of a certain commodity. Some of my variables (such as price itself, as well as inflation, and taxation) don't have any seasonalities. ...
Thomas's user avatar
  • 11
0 votes
0 answers
18 views

How to estimate the effect of time series on a variable sampled irregularly and much less frequently for multiple subjects

I’ve been struggling for several days to find the proper statistical analysis tools for my problem, and I’m hoping for some valuable tips and insight from the internet. I’ve read up on ARIMA(X), ...
timeSerious's user avatar
0 votes
0 answers
29 views

Can LASSO still perform regularization on summarized data?

Currently, we are trying to predict future revenue from existing users. We use the revenue collected after 14 days of membership to predict 3 year membership. We train the model and make predictions ...
Demetri Pananos's user avatar
0 votes
0 answers
9 views

Prognosis modelling data set

I want to externally validate and update a previously developed and published model (Model A) in my dataset. In the case of poor performance, I would like to develop a new model for the same outcome (...
imunicorn's user avatar
0 votes
1 answer
40 views

Predicted cloud of points is tilted compared to the diagonal [duplicate]

I built a model with 2 variables, i found that the predicted values for the test set did not align with their real values, so i predicted instead with the data which were used to train the model. I ...
Renaud Bied-charreton's user avatar
1 vote
0 answers
66 views

How to create representative training, validation, and test sets when working with time series data?

In my application, I am working with a relatively long time series of daily market index percentage returns (many years) and am trying to model the dependence structure of the returns from a pure time ...
QMath's user avatar
  • 185
2 votes
0 answers
27 views

Functional Principal Component Analysis - Explaining Functional Principal Component Scores

I was wondering if someone can help with explaining Functional Principal Component Scores? I am working with a dataset which reflects participants in a weight loss management trial (longitudinal data)....
Data_Science_Mick's user avatar
3 votes
2 answers
85 views

Why do we seek to find a model that approximates all data instead of finding a function that fits all data?

I tried to learn the principle idea of regressions. As I understand, the aim is to find a model that represents the relation between the x numbers and the y numbers, so that we can understand the ...
Magician's user avatar
0 votes
0 answers
24 views

What's the Issue with Modeling Admissions as Multinomial?

I am reading Chapter 4 (Testing and Confidence Regions: Basic Theory) of Mathematical Statistics by Bickel and Doksum. In Example 4.1.1, the authors use the study of sex bias in graduate admissions at ...
Extrava's user avatar
  • 123
0 votes
0 answers
33 views

Is there a statistical model for the problem I want to solve?

I have two time series features, $A$ and $B$: both returns. When $A$ is at 0 (as it is for the vast majority of the dates), you cannot say anything about $B$; it's just noise. However, when $A$ ...
Bepop's user avatar
  • 197
1 vote
0 answers
14 views

Develop a global model for multiple objects

First of all, i don't know if "global model" is the right name for this. The goal is to develop one model to predict the solar energy of PV-Systems in an energy management system for the ...
MBC_222's user avatar
  • 11
1 vote
1 answer
20 views

Geometric meaning of cross product of a design matrix and model coefficients

I am trying to understand more about the geometry of linear modeling. Take for example an experiment with one categorical predictor having two levels and a numerical response. If two data points are ...
Chris Science's user avatar
1 vote
1 answer
9 views

Difficulties to decide on model formulation and to identify dependent and independent variables

Imagine a disease whose exact mechanisms are unknown and can lead to different phenotypes. Although there likely is a continuum between the different phenotypes, these are categorized (as ordinal ...
jkd's user avatar
  • 344
1 vote
0 answers
20 views

Gaussian white noise model in application

I am interested in applications (to data) of non-parametric statistics, and my question concerned the Gaussian white noise model defined by, $$ X_{t_1, \ldots, t_d}=f\left(t_1, \ldots, t_d\right) d ...
BabaUtah's user avatar
1 vote
0 answers
31 views

Using both Random Forest and RFE for Feature Selection and Dimensionality Reduction [closed]

I am using random forest classifier currently to do feature selection for a balanced dataset of about 18K rows and 7050 features. I recognize this is a lot of features. I am thinking of using random ...
Jimbo's user avatar
  • 11
0 votes
0 answers
13 views

What kind of model do I use to calculate the probability of an observation falling into each of several categories?

First off, I don't have a heavy data science background, so if I have the completely wrong approach for this, a pointer in the right direction would be appreciated. I am trying to build a series of ...
Wolff's user avatar
  • 141
0 votes
0 answers
18 views

Defining parameters so that they obey multiple constraints

I'd like to define parameters $\beta_i$ for $i=1,\ldots,I$ for a problem so that they automatically obey some constraints. The constraints are: $\sum_{i=1,\ldots,I} w_i \beta_i = c_1$ and $\sum_{i=1,\...
Björn's user avatar
  • 29.3k
0 votes
0 answers
29 views

Time series: how to model the impact of exogenous variable?

Disclaimer: this is a more theoretical question than a practical one. I am trying to predict the revenues of a company (so I can't post the data) using the past 4 years data (monthly) and exogenous ...
KeyPi's user avatar
  • 101
0 votes
0 answers
37 views

Dirichlet Regression output and using the calculated coefficients in regression model

I am very new to Dirichlet Regression and trying to make sense of the output and the regression coefficients. I am doing a biomass study and have tested the following variables (DBHH, DBH + H, DBH and ...
Otto_P's user avatar
  • 1
0 votes
0 answers
25 views

Non-Linear data fitting for kinetic data

I am trying to fit a two_site model which I derived, (basically, rate as a function of pressures of reactants and products) to my experimental data. Just want to check if the logic below is fine, ...
Suyash Sachin Damir's user avatar
0 votes
0 answers
25 views

Can confounders be controlled for in an Interrupted time series and when should outcomes be modeled as binary rather than aggregated rates?

I just learned about interrupted time series and have a few questions about them. Say I have a dataset of individual patients and I want to compare their monthly rates of getting a certain lab test ...
M. Yates's user avatar
0 votes
0 answers
20 views

R codes for Bayesian Credible Intervals and p-values for a Additive Bayesian Network model

I am using Additive Bayesian Network (ABN) to model the relationship between cereal crop production and climate variables and want to generate credible intervals and p-values for my model coefficients....
Kofi Willie's user avatar
0 votes
0 answers
29 views

Is there a way to estimate dependence between an observed variable and an unobserved variable?

I'm attempting to do some open-ended exploratory analysis/model building on real-valued time series data. I am explicitly not assuming that all elements of my class of models are linear in the lagged ...
QMath's user avatar
  • 185
0 votes
0 answers
24 views

How do i write equation for a lognormal graph if I have 100 points on the graph that clearly have shape of a lognormal graph and the x any are known?

Days 1 to 100 are along the x-axis. The y-axis has the PDF values such that the area under the graph is 1. (x,y) is known for each point. The points line up in a near perfect lognormal distribution ...
Joseph's user avatar
  • 11
1 vote
0 answers
27 views

Should an offset be removed if the variable doesn't seem to have much influence?

This is about a joint project with a subject matter expert, and I'm the statistician. I will not explain the background and simplify the statistics problem as the background is rather complicated. The ...
Christian Hennig's user avatar
1 vote
0 answers
17 views

PLSR: trait vs spectroscopic data gives very low R2 on plsr model in R

here is the sample data. I have spectroscopy data as X-variables (from X1 to X80) and corresponding Y variable. I need to run plsr model in R using "pls" package. There are two sheets. In ...
MGD's user avatar
  • 11
0 votes
0 answers
29 views

Quantile regression and prediction estimate

I am working on a personal project and I've been wondering about something. I'm basically taking some loan data from kaggle, analysing/cleaning it, then trying to apply machine/deep learning models to ...
FaresDjerourou's user avatar
0 votes
0 answers
13 views

ML (NN) model for a physical process (layer movement in ensilage)

The problem is, I have a several silages with several layers of some substance in them (e.g. coal). Each layer has its own physical/chemical properties (concentration of element, X). Concentration of ...
DDR's user avatar
  • 111
0 votes
0 answers
10 views

Statistical models or problems equipped with methods that guarantee optimal or near solutions in finite time

What are some statistical models or problems that have computationally tractable solutions and are guaranteed to be optimal or near optimal? By computationally tractable, I mean finite time and ...
user's user avatar
  • 2,386
1 vote
0 answers
43 views

Popcorn- what's the best way to compare the fit non-linear models for data that is not very complex?

I popped 7 bags of popcorn in 30 second increments and found the percentage of popped kernels in each time frame. I manipulated this data with sinusoidal, exponential, and cubic models. There is lots ...
robert's user avatar
  • 11
11 votes
4 answers
2k views

Methodology for Reconciling "all models are wrong ..." with Pursuit of a "Truer" Model?

When attempting to model various phenomena, in practice, our model will usually be "wrong", but can still provide us with useful insights/predictions. What methodologies can we follow to ...
QMath's user avatar
  • 185
4 votes
2 answers
278 views

Modeling continuous interaction effect with very small sample

I have a very small data set (n = 12) of high level athletes each with a set of measurements representing technique (5 variables) and a set of measurements representing strength (3 variables), and I ...
user avatar
0 votes
0 answers
31 views

How to model breakage events for electronic devices using a Weibull distribution?

I got age data (in years) of gas heatings from a random sample of about N=700 participants. So participants were asked how old their gas heating is in years. Now I like to model that age distribution (...
Dirk's user avatar
  • 3
0 votes
0 answers
22 views

Why am I missing the result of a smoothing effect in a GAMM while interpreting results from summary command

I have been using GAMMs to analyze time series data and I have included a smoothing term (hour of day by season) and I can't seem to find the results for the winter season. I have the proper ...
Kevin Short's user avatar
0 votes
0 answers
17 views

How to interpret smoothing effects in the summary output of a generalised additive mixed effect model GAMM

I have been using GAMMs to analyse time series data and I have included a smoothing term (hour of day by season) and I cant seem to find the results for the winter season. I have the proper ...
user avatar
0 votes
0 answers
13 views

GARCH fit to the residuals of AR/ARMA mean equation previously fitted

Suppose I have an ARMA (p,q) (let it be ARMA (2,2)) fitted to my original returns series and have the residuals of said ARMA model extracted. Next, it is my understanding that I need to fit a GARCH ...
Ghada's user avatar
  • 1
0 votes
1 answer
36 views

Feature importance/model exploration in a large survival model

I've built very-well performing survival model (weibull proportional hazards model for interval censored data, modelled with IcenReg) with many covariates, some ...
Wojty's user avatar
  • 53
5 votes
1 answer
201 views

Ranking of multiple raters with partially overlapping assessments

Ranking of Raters and Cases I am looking for a method for ranking a set of raters with partially overlapping binary assessments. I also want to rank the cases according to their difficulty. Context I ...
Filip's user avatar
  • 59
9 votes
1 answer
643 views

What do I do when a false negative is far more expensive than a false positive?

I'm not sure how to 'responsibly' balance my model to account for this. I could predict a probability and give that to the business ('predict_proba' in SKlearn) but experience in the past has thought ...
Cdl's user avatar
  • 193
0 votes
0 answers
19 views

Effects of multiple non-independent treatments

I need to run an analysis of efficiency of automated recommendations for call-centre agents on how to handle customer support tickets. It works like that: A ticket get 0, 1 or more recommendations ...
DVS's user avatar
  • 1
2 votes
2 answers
146 views

How to deal with noisy observation in Survival Analysis

I'm new to Survival Analysis. Usually in survival analysis, we want to model the survival function progress w.r.t time. This is normally done through Cox model, or KM-model within a specific time ...
Wakeme UpNow's user avatar

1
2 3 4 5
51