All Questions

0
votes
0answers
3 views

What is a technical term for a model that takes as input the output of multiple other models?

I have 60+ models (could be basic if-else rules or complex neural networks) that describe population risk. Each of the models has at least one output (often more). What is the technical term for the ...
0
votes
2answers
8 views

300k observations, significant results but very low adjusted r-squared

The model I have for the regression is in accordance with literature. I have about 300k observations and 25 dependent variables. When I run OLS regression, the output shows a couple of significant ...
0
votes
0answers
4 views

Monte Carlo Simulation using R: Problem with sorting and significance

I am trying to implement the following statistical test using Monte-Carlo simulation. This method is based on the following paper: https://journals.ametsoc.org/doi/full/10.1175/JCLI4217.1 Details: ...
0
votes
0answers
2 views

Is Stratified K Fold CV Needed when Estimator implements Balanced Class Weight?

I am working on a classification task with an imbalanced dataset. I am using Sklearn's ensemble RandomForestClassifier and set its class weight to Balanced. My question is, when I then GridSearch it, ...
0
votes
0answers
5 views

SVM in the classification layer of a Feedforward neural network

I want to use SVM in the classification layer of a 2 layer feedforward neural network. Need guidance from the community on how to approach this problem. This involves capturing the features from the ...
0
votes
1answer
9 views

Generalized linear model problem: how do we estimate the parameter $\lambda$ from the Poisson distribution?

To begin with, I am going to show off the context from which the problem arises. Consider that we are concerned with counting the number of organisms of particular interest within some sample. One ...
0
votes
0answers
2 views

UMVUE of the probability a Poisson R.V is odd?

Problem: Let $X_i \sim Pois(\lambda)$. Find the UMVUE of the probability that $X_1$ is odd. My attempt: I don't think there's any obvious unbiased estimator to use conditioning. So instead I write $...
0
votes
0answers
7 views

If I wanted to see if their was a difference between two groups and paired each person with someone of similar age what test would I do?

I was thinking of doing a paired samples t-test, since we are pairing specific individuals from each group with each other. Is this correct?
0
votes
0answers
7 views

Can you perform statistical inference in a multi-armed bandit scenario?

Statistical inference is the process of using data analysis to deduce properties of an underlying probability distribution. Multi-armed bandit is a technique to try different options and sample more ...
0
votes
0answers
6 views

Confusion about computation of average precision

I am trying to learn what AP (average precision) means and I came across this page: https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52 Here is the given formula: ...
0
votes
0answers
6 views

Fitness metrics for a classification problem

I am new into ML and trying to play around and find an expression that yields a binary (Yes/No) output for a simple classification problem (with 7 inputs). For one solution (obtained using genetic ...
0
votes
0answers
4 views

How to interpret sensitivity index?

I am trying to do risk analysis on the statistical models we created. I used the sobol sensitivity analysis method to calculate the sensitivity indexes, using a python package SALib. Now how do I ...
0
votes
0answers
7 views

Extract covariate adjusted dependent variable from linear regression model?

I want to adjust a continuous (dependent) variable for two independent variables (1 categorical, 1 continuous) and then use the residuals + intercept for plotting by another categorical variable. Is ...
0
votes
0answers
11 views

Different results when conducting Anova using R

Trying to figure out what I am doing wrong here: The below is my data set ...
0
votes
0answers
8 views

How can I analyse simmilarities/differences of one nominal variable of two different groups?

I have two different samples. I want to measure how similar is each individual of the first group with the second group in terms of Var 1 (nominal not ordered), given that they are both categorized ...
0
votes
3answers
38 views

What is covariate?

I'm confused with this term: covariate. What is it? Is is just the observed outcomes of some random variables that contain information that could help us enhance our prediction of another random ...
0
votes
1answer
13 views

student attendance prediction

Let's say I wanted to be able to predict, from game to game, what the total student attendance would be at an average college football game, where would I even begin? I would have access to ...
0
votes
0answers
10 views

Covariance matrix and persistence of excitation of input

Assume that a discrete-time system can be described by the following state-space equations $x(k+1)=Ax(k)+Bu(k)+w(k)$ where the input signal $u(k)$ is stationary and ergodic with $\mathbf{E}[u(k)]=0$....
0
votes
0answers
4 views

Does someone has a good implementation of categorical focal loss? [on hold]

Does someone has a good implementation of categorical focal loss ? I'm not sure of existing package. Thanks
0
votes
0answers
17 views

Alternatives to The Statistical Sleuth?

I am looking for a simple and concise book on statistics (t-test, ANOVA and all its variants, liner regression, etc.), centered on data analysis. I am not interested in theory or proofs, but just want ...
0
votes
0answers
14 views

Image dimension reduction using pca

I have a 3000 of images each of which has shape of (200,180,3). I want to reduce their dimension. Therefore, I use PCA, I want to do it firstly without using any library. The first problem I camed ...
1
vote
2answers
15 views

Which analysis should I use

I am comparing data from two cohorts of patients that underwent a surgical procedure: Group 1 (2013-2015, n=157) and Group 2 (2016-2018, n=146). In both cohorts, I have patients that had survived and ...
0
votes
0answers
16 views

Analytic solution to optimization involving step function

Say I've observed some samples of two random IID variables, $\{x_i\}$ and $\{y_i\}$. Is there an analytic solution to the following optimization problem? $$ \underset{a}{\mathrm{argmax}} \sum_i \...
0
votes
0answers
7 views

What statistical parameters to use to spatially compare rasters?

I have at my disposal four models (Model 1, Model2, Model3, Model4) that simulate a parameter (the density of drainage) in a 7.5 'grid (about 14 km * 14 km) on the same domain (watershed) confer ...
0
votes
0answers
4 views

Randomize dataset for Restricted Boltmann Machines

Suppose I want to train a RBM (or even a DBN architecture) and then fine-tune the parameter training a Feedforward NN. In my case the dataset is composed of time series, so in principle there is a ...
0
votes
1answer
11 views

Name for decomposition of joint probability into product of conditional probabilities

Consider a set of events $A_1, \dots, A_n$. By definition we have $$\mathbb{P}[A_1 \cap (A_2 \cap \dots \cap A_n)] = \mathbb{P} [(A_2 \cap \dots \cap A_n) | A_1] \mathbb{P}(A_1).$$ Applying the ...
0
votes
0answers
10 views

Functional Regression [on hold]

How to fit the functional regression with both independent and dependent variables are functional. how i can fit functional regression with tow independent variables which are both are functional. how ...
0
votes
0answers
4 views

Comparing regression/correlation values within-subjects? Repeated Measures ANOVA afters Fishers Transform?

I was interested in the most appropriate way to assess whether there is a difference in the R values from a regression analysis in a repeated measures design. I transformed my R values using a ...
1
vote
0answers
12 views

What is the effect of balanced class weights on logistic regression coefficients?

I'm specifically using sklearn's LogisticRegression on my unbalanced dataset, which has around 97% negative responses and 3% ...
0
votes
0answers
8 views

Standardize prevalence and incidence rates calculation in R

There is a cohort study of 500,000 individuals with over 70 conditions. For each condition, I need to calculate Annual standardized prevalence, with time since birth as the underlying time scale ...
0
votes
1answer
22 views

Why would you express a time series as deviations from the mean?

I have been trying to learn time series analysis on my own and in a textbook I came across the idea that one could express each data point in a stationary time series as deviations from the mean in ...
0
votes
0answers
21 views

Tjur's R^2 for Logistic Regression

I'm designing an experiment and plan to use Tjur's ${R}^2$, denoted $D$, to evaluate some logistic regressions. He defines the value in these three equivalent ways: $D = \bar{\hat\pi_1} - \bar{\hat\...
0
votes
1answer
20 views

Could someone explain these ACF and PACF plots for what I think is a lag1 series?

I am trying to get my head around simple time series analysis. I think this R code shows my confusion. ...
0
votes
0answers
5 views

Transformer based decoding

Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder ...
1
vote
0answers
7 views

FE and clustered standard errors for a panel with firms, industries and years

I have an unbalanced panel of firms, years and industries. (1) At the moment I am running a panel regression with years and industries FE. My understanding is that I am counting for any omitted ...
1
vote
0answers
22 views

Using a time series analysis to explain an event

I have hourly data for sand dune temperature at various depths as well as air temperature from summer 2015 to winter 2019. I have the day on which we saw our study species (one species) for the first ...
0
votes
0answers
22 views

Terminology: “L1 regularization” even if I'm using mean instead of sum? [duplicate]

In my loss function I'm using the mean of the log-cosh error between the predictions and targets, as well as an additional regularization term that scales as the mean of the absolute value of another ...
0
votes
1answer
23 views

Variance vs Variation

So far both terms have always meant the same thing for me, however, I started wondering, do they mean the same thing? The first 3 principal components explain 80% of the variance. The first 3 ...
1
vote
1answer
14 views

Can sampling from a truncation of a random variable, rather than the original variable be more Blackwell-informative?

Suppose you are interested in finding the mean of a random variable. You have some prior belief of it and before sampling 1 observation, you can decide whether to sample from the original random ...
0
votes
0answers
4 views

How to get PC values for individual datapoints using the prcomp command for PCA in R-Studio [on hold]

I just started using R-studio so please excuse me if this question has a glaringly obvious answer. I have been trying to carry out a principal component analysis using prcomp. Although everything ...
0
votes
0answers
4 views

econometrics f-test HELP [on hold]

`Hi, i am having problems with this econometrics question, urgent help is needed!!! the model of observations t=1,2 .....,T is given by: yt=B1+B2x2t+B3x3t+B4x4t+ut an investigator wants to test ...
0
votes
0answers
6 views

Optimal pandas dataframe size for seasonal ARIMA predictions over multiple timeframes

For backstory, please see this related question: Q3: How many rows of data should I keep in my respective timeframe for accurate ARIMA modeling (using SARIMAX)? An example of the p,d,q's of a 1H ...
0
votes
1answer
19 views

Interpreting dummy variable interaction terms

I am attempting to model monthly retail electricity sales. To account for both the effects of seasonality and weather, I created an interaction term by multiplying 12 monthly dummy variables by the ...
0
votes
0answers
7 views

Generating nested models for order selection tests [on hold]

I have created a parametric additive model for the median house price (as the response) and with the number of tax forms (x1) and the number of healthcare facilities (x2) as my covariates. I should ...
0
votes
1answer
9 views

How to interpret a high sensitivity and low specificity using svm classifier?

I want to have your opinion on the performance of my SVM classifier (k-fold cross validated): Classification of class1(n=45)/class2(n=86) Accuracy: 65.4% Sensitivity: 88.2% Specificity: 22.2% AUC:...
1
vote
1answer
14 views

How to implement constraints in a neural network for a regression problem?

Let's say I have different sensors in an engine, and I make a neural net which predicts the engine's temperature given different operating conditions measured by the sensors. I happen to know the ...
0
votes
0answers
8 views

Estimate the restricted Nested Logit with R [on hold]

I find it very difficult to estimate the model proposed in the post title; in particular: I have done the following code ...
0
votes
0answers
20 views

Indentify subset of data that follow linear regression

I have a dataset that first NL points follow a linear model and the next Na points follow a log model. My problem is that I do not know the value of NL and I want to get it from the dataset. I am ...
0
votes
0answers
5 views

Scoring the difference between a family of distributions and a test distribution

Let's suppose we have a random model that I can sample to generate distributions of a certain 1D variable. I want to score the distance of a test distribution to the model in question. The distance ...
0
votes
0answers
19 views

Interactions terms and the dummy variables

I am attempting to model monthly retail electricity sales. To account for both the effects of seasonality and weather, I created an interaction term by multiplying 12 monthly dummy variables by the ...

15 30 50 per page