Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [model]

A statistical model is a formalization of relationships between variables in the form of mathematical equations. The model is statistical as the variables are not deterministically but stochastically related.

0
votes
0answers
11 views

R: How can I make a graph showing the distribution of my model vs the actual distribution by decile? [on hold]

At my company, we currently use a program that spits out the below graph for model fit. and I'm trying to replicate it in R. The graph splits it into deciles, then shows a line of the actual ...
0
votes
1answer
20 views

Differences in calibration plots for machine learning models

I'm using machine learning methods in R for descriptive regression modelling of a small dataset. I have fit random forest (randomForest), unbiased random forest (cforest) and boosted regression trees (...
0
votes
0answers
13 views

Joint distribution of a two part model

Let $ Y $ be a random variable defined on $ (0, +\infty) $. In a univariate two part model, the distribution of $ Y $ is defined as follows \begin{equation*} g ( y_i ) = \left\{ \begin{array} { ...
-1
votes
0answers
13 views

How to test the homoskedacity using GQ test, Interpret R2 with VIF, Calculate Adjusted R-squared [closed]

Can you please show me with some explanations how to test the homoskedacity using GQ test, calculate and interpret R2 with VIF for a model, Calculate Adjusted R-squared from the pictures from this ...
0
votes
0answers
22 views

Generalising AIC results over multiple samples

This is slightly related to my previous question (AIC Calculation using log likelihood) Though, I think now I am actually clear as to what I am asking. I am modelling activity of cells, I have data ...
0
votes
0answers
10 views

Quasipoisson model variable selection and find best model

I am running a Quasipoisson model in R with a lot of variables. This is my outcome: I want to find out which variables have an influence on the dormouse abundance (number of nests). After doing the ...
0
votes
0answers
7 views

Stacking Final Model Development After Cross Validation

CV (or Nested CV) are normally done to evaluate and compare different ML algorithms as part of model development and evaluation phases. Once these stages are complete, one normally develops the final ...
0
votes
0answers
3 views

planned comparisons across a continuous gradient

I am analyzing data on stream community before and after an invasive species removal event and have a dataset of dissimilarity values comparing the stream community at each time point in the dataset ...
1
vote
0answers
22 views

Quantifying the observed vs predicted in relation to identity line [duplicate]

I have a model and observed data. I would like to plot the relationship of observed vs predicted values. Predicted values would be generated using the model with optimised parameters. I am ...
9
votes
7answers
258 views
+50

Avoiding social discrimination in model building

I have questions inspired from the Amazon recent recruitment scandal, where they were accused of discrimination against women in their recruitment process. More info here: Amazon.com Inc's machine-...
1
vote
1answer
27 views

Model Quasipoisson interpretation and validation

I am currently doing my Master thesis with evaluating my results in R. I am stuck on my analysis of my glm with quasipoisson. I am analysing influencing variables on the dormouse abundance in 2 types ...
0
votes
0answers
12 views

Should oversampling/undersampling be applied only during CV or also for final model creation?

I am dealing with a highly class imbalanced dataset and am going to try oversampling and see how my nested CV is affected when comparing algorithms. When it comes to model finalization, should I ...
0
votes
0answers
27 views

Best way to model the dependency of these two random variables (copula?)

I'm modelling the joint PDF of two variables that looks like this , where vt and vr are the random variables. The dashed line shows the joint pdf assuming they are independent (the product of its ...
0
votes
1answer
59 views

Does 'high' bias really exist when we are unsure how to quantify the irreducible error when making a machine learning model?

My friend just exited an interview where he had used the terms "underfitting" and "overfitting" as equivalents to the bias/variance tradeoff, but he says his interviewers were looking more for an ...
1
vote
0answers
20 views

Autocovariance and autocorrelation function of AR(1) process

I'm preparing the exam about AR models, precisely I have this exercise which I have some issues with points "d" and "e". My try was: Knowing that $W_t=X_t-X_{t-1}$, $h=1$ so: d) $\gamma\left(1\right)...
0
votes
0answers
30 views

multivariate linear regression without b_0 [duplicate]

I created a multivariate regression following the scheme $$y = \beta_0 + \sum^n_{i=1}\beta_i*x_i$$ and got an average deviation ofaround 5%. When I tried the regression without the $\beta_0$ I got a ...
0
votes
0answers
11 views

Making a more reactive model

I am working on a model to predict quarterly values. I am running into an issue where my back data has extreme trends and my model (I am using a Holt-Winters model) seems to be taking these old values ...
1
vote
0answers
29 views

How can one verify distributional assumptions for testing procedures, if only one draw from each distribution is available

I am currently dealing with a situation in which the Wilcoxon signed rank test (WSRT) is a possible candidate to be applied to the data. So let`s say we have $n$ pairs of Random Variables $(X_i, Y_i)$ ...
0
votes
0answers
11 views

Modelling my own cost function for skewed distributions

A while ago I worked in a model used for fraud detection, as that involved a very asymmetric distribution, I used F1-score as my loss function. However, I cannot stop thinking about it, and I feel F1-...
1
vote
1answer
18 views

perform BIC to compare the full model and the reduced model

When we were doing the linear regression, let's say the reduced model $y=\beta_0+\beta_1x_1+\beta_2x_2+\epsilon$ is the true model (i.e. we were using this model to generate data), then how can we ...
1
vote
0answers
20 views

What is a good model for learning power equation against frequency for a chip

Here's my problem scenario:- I have to come up with a power equation as a function of frequency. The plot fits well with a higher order polynomial (4th or 6th) :- $$Power = \theta_0 + \theta_1 fr^1 + ...
0
votes
0answers
16 views

GPower Sample Size Error with Large ANCOVA Model

I am trying to do a power calculation that with a very large ANCOVA model but am getting an error because my number of groups is too large. I have 14 binary variables, two 4 factor variables, a 5 ...
3
votes
1answer
51 views

How is an RNN (or any neural network) a parametric model?

I'm going through this paper A Multi-Horizon Quantile Recurrent Forecaster. The authors state that: 3.1. Loss Function In Quantile Regression, models are trained to minimize the total ...
0
votes
0answers
27 views

eigenvalues and multiclionarity

I want to know if I have a problem with multicollinearity based on this table. Could you please help with this
2
votes
1answer
40 views

What is the connection between Bayesian Model Averaging and SSVS?

What exactly is the difference between Bayesian Model Averaging (BMA) and the Stochastic Search Variable Selection (SSVS) prior when we talk about linear regression models? The SSVS prior is used ...
0
votes
1answer
24 views

Appropriate model for repeated measures in R (Effect of treatment)

I would like some advice on what statistical model I can choose and how to implement in R. I have 2 groups of high blood pressure patients (each group consists of 40 subjects ), a treatment group (...
0
votes
1answer
26 views

Propensity Score Matching on demographic baseline

A client asks for a PSM on gender for their big dataset of >10000 cases. About 20 variables are supposed to be included, most of them binomial. They hypothesize, that a certain treatment has worse ...
0
votes
0answers
13 views

What are some methods to analyse large sets of data in effective and efficient ways?

I have large amounts of data for clients of a transport provider (think similar to a taxi) in and around New York City. The kind of information I have is: What area they are travelling to and from. ...
0
votes
1answer
26 views

How to consider a variable that has both random and fixed values?

I have an experiment in which we want to measure the effect of a medication on mean arterial pressure (MAP). We have 3 conditions for each patient: placebo (no treatment), fixed standard dosis (100mg) ...
0
votes
0answers
23 views

Can I merge variables from likert scale and what regression should I use?

I am working in R and currently working on my thesis where I am analyzing if some factors have impact total money spend on virtual items. So I would like to use total money spend as dependent variable ...
1
vote
2answers
32 views

LM Model Assumptions: Transforming Data in R using log()

I have a dataset in which I am trying to fit a model for: model <- lm(expression_fold~distance, data = pairwise_sub) However the data set is heteroscedastic, ...
1
vote
1answer
41 views

Negative Binomial Model: Fixed vs Random Effects

How to choose between Fixed and Random Effects in panel negative binomial model? Is Hausman test valid for this?
1
vote
3answers
153 views

To overfit, or not to overfit, that's the question

I hope this is not a stupid question. Let us say I have a data generation process that is quite stationary and I do not care about arriving at generalizable knowledge but more about accurate ...
0
votes
0answers
13 views

When to update a model

This is an open question that I have - when should a model be updated? In practice, the modelling process could take a number of weeks/months to update and I would just like to know what some of your ...
5
votes
0answers
44 views

Why does “mixtools” return the model with highest AIC as the “winner” if lower AIC is better?

Mixtools package is used to fit mixtures of normal/regressions. The package documentation is given here The regmixmodel.sel fits the mixture model for varying ...
3
votes
2answers
51 views

Suppose is is stated that $X_i \sim \mathcal{P}_X$ is a non-parametric model. What does this actually imply?

I read in a book that sometimes one can specify a non-parametric model. It stated that: $X_i \sim \mathcal{P}_X$ for $i = \{1, \ldots, n\}$ was a non-parametric model. My understanding of a non-...
0
votes
0answers
11 views

Model selection for mixture of regressions - what order?

Suppose we are fitting a mixture of regressions. That is, $y_i$ is assumed to come from a mixture $$f(y) = \sum_k \pi_k f_N(y|\mu_k, \sigma_k )$$ Where $$\mu_k = \beta_k^T x$$ and $f_N$ are ...
1
vote
0answers
21 views

Optimization of revenue (with multiple products and constraints)

Have a data science problem, but I am not sure how I can approach the problem. Problem: I want to optimize my revenue, with my data. Only constraint: stock should be >0 (otherwise we will not sell ...
0
votes
0answers
10 views

Truncating covariates if linear relationship breaks down for large values?

Suppose you fit a model $$y_i = \beta x_i + \epsilon_i$$ Where you strongly believe this linear relationship holds for the vast majority of observations. Suppose, however, that a small number of ...
0
votes
1answer
18 views

volume generation with GAN

I'm not a GAN expert, but I have a problem and I would like to understand if GAN could help me in some way. Essentially my problem is to convert a 3D grayscaled volume in another 3D grayscaled volume, ...
0
votes
0answers
6 views

Compare the performance of a new model with the existing model

The challenge for me here is that I do not know the test/train datasets of the EXISTING model but I have the model with me (it is a logistic regression equation). I believe the NEW model's AUC on ...
0
votes
0answers
11 views

Bayesian information criterion (BIC) on KDE?

Consider two datasets, $X$ and $Y$. Both have 2 dimensions with $a$ and $b$ samples respectively. I would like to test whether one kernel density estimate (KDE) on the concatenated data ($XY$, shape $...
0
votes
0answers
14 views

Use or not use a count variable when data is static

I have a dataset of objects (e.g. toys) for which we keep over the years a count of specific events (e.g. minor incidents). Based on this count, I'm trying to predict for each toy if an accident is ...
4
votes
1answer
50 views

How to test whether 2 prediction intervals are statistically different?

I've been struggling with this for a while now, hopefully someone will know how to help me :) Here it is : 1) I'm using a linear mixed effects model on longitudinal data (biological values of many ...
0
votes
0answers
20 views

standard error- residuals

assume I am predicting home runs, assume all player bat the same number of times, so we can do this by total home runs, and not home run rate) from a player based on past experience. I have a linear ...
1
vote
0answers
20 views

R: Is there a method similar to PCA that incorperates dependence

Background We currently applied PCA to a set of variables, and noticed that our dataset actually contains two "motifs". To explain, let say we have the variables ...
1
vote
0answers
37 views

Different model result stats::arima and dynlm

I am calculating an autoregressive model with two different libraries (stats and dynlm). Attached you can find the code and the data. I am using in both libraries the same methodology (least squares). ...
0
votes
0answers
56 views

Germination statistics

To start, I have to apologize for my lack of knowledge in statistics. I have stumbled on to the problem that I find hard to solve on my own. I conducted the experiment where I placed bunch of seeds ...
0
votes
0answers
10 views

Determining the Relationship Between Monte Carlo Breaks and Model Volatility

I'm looking for a statistical test to understand the relationship (if any) between the model volatilities of a stochastic process, and the occurrence of a'break', defined as an instance when an ...
0
votes
0answers
29 views

Non-orthogonal experimental design and model selection

I am working on designing some chemical experiments, with the goal (for now) to optimize reaction yield. I intend to use principal component scores in order to investigate solvents, Lewis acids etc. ...