All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Can I perform hyperparameter tuning inside K-fold CV?

I am performing lease one subject out cross-validation, using one subject as the independent final test set to get the performance of my model. Can I perform hyperparameter tuning inside each of my K-...
user avatar
0 votes
0 answers
6 views

Solve the EM algorithm of the Gaussian mixture model

Considering a mixture of two normally distributed random variables $Y_1$ and $Y_2$ and considering the linear combination of the two: $$ \begin{gathered} Y_{1} \sim N\left(\mu_{1}, \sigma_{1}^{2}\...
user avatar
  • 599
0 votes
0 answers
5 views

General Closed Form and dispersion parameter of the Expected Maximum of i.i.d Gumbel Variables

I would like to know the general closed form of the expected maximum of i.i.d Gumbel variables. I found this onlie: Expectation of the Maximum of iid Gumbel Variables In the linked page, it shows the ...
user avatar
0 votes
0 answers
13 views

Doubt regarding inverse CDF/quantile function or qpois in R

Let's say I have a poisson with mean 9.29 (lambda). The probability of random variable being less than equal to 1 and 2 are "0.0009502101" and "0.004935002" respectively (i.e. P[X&...
user avatar
0 votes
0 answers
10 views

Classifying an Event Based on Time of Day

My problem is very simple: Say I am running a hotel, and I want to make an educated guess on whether a guest is a business or leisure traveler. The guest does not tell me if they are business or ...
user avatar
  • 101
0 votes
0 answers
6 views

Discrepancy in degrees of freedom from R svyglm vs glm

I fitted a Poisson model using svyglm in R. The null and residual deviances from the svyglm model are as expected. For the degrees of freedom however, I get confusing results. With a sample size of n=...
user avatar
  • 1
0 votes
0 answers
3 views

Assigning weights to independent variables for a composite metric

I am trying to evaluate merchant experience for a company. I have a set of metrics like success rate, service tickets, ticket sla breaches etc as independent variables. Merchant experience would be my ...
user avatar
0 votes
0 answers
4 views

Compute two proportion tests (with three variables?)

I want to make sure I'm running these proportion tests correctly to show that there are differences in outcomes for three groups. I am working off of this guide, except I have an additional group. ...
user avatar
  • 157
0 votes
0 answers
13 views

Assumptions in logistic regression for applying LRT and AIC criteria

I have a question. Do the following assumptions in logistic regression Linearity between the log-odds and the continuous covariates Non multicollinearity Absence of outliers need to be satisfied ...
user avatar
0 votes
0 answers
13 views

What are the theorems behind this variance formula?

I saw below formula to derive the variance calculation for $Var(\widehat{\delta_{cdx+}})$. I am wondering what theory has been applied in the second and third rows.
user avatar
  • 1,288
1 vote
1 answer
12 views

Using conjugate priors to estimate the posterior distribution of a proportion of a region composed by subregions

Let's say I have a region that is divided into 3 subregions. In each subregion, I run ~90-110 randomly allocated surveys asking a binary question. I want to know if the way that I am estimating the ...
user avatar
0 votes
0 answers
7 views

How to forecast macro trend by multiple index by LSTM model? [closed]

I just start exploring machine learning world. I want to try predicting the macro economic trend by grouping different index futures. After reading many article, I have came up 2 approaches below. May ...
user avatar
  • 101
0 votes
0 answers
12 views

Steps to build a classifier [closed]

Doing my research into classifiers, i found this topic: Should I first oversample or standardize (when cross-validating on imbalanced data)?_ I got some nice advice in there, but since i have more ...
user avatar
  • 101
0 votes
0 answers
10 views

How to perform image classification in a dataset of images with heterogeneous sizes?

I have a dataset of images with very different sizes (ranging from 100X100 pixels to 5000X1000 pixels) and aspect ratios. I want to use neural networks for dealing with this problem. Is there any ...
user avatar
  • 412
0 votes
1 answer
14 views

ARIMA - GARCH or AR-GARCH

I am looking at equity returns and they are not stationary at level. So i take the 1st difference to make them stationary. My GARCH(1,1) model is modeled using an AR(1) parsimonious model. But, since ...
user avatar
0 votes
0 answers
3 views

Determine cause of poor concordance index?

I have a survival dataset which I’ve experimented with to create several Cox PH models using different techniques (lasso, forward selection, backwards elimination etc), however no matter which ...
user avatar
0 votes
0 answers
12 views

MatchIt in R option for euclidean distance matching [closed]

I read in MatchIt that there is option to specify "euclidean" for distance: https://cran.r-project.org/web/packages/MatchIt/MatchIt.pdf. However when I put distance = "euclidean" ...
user avatar
0 votes
0 answers
9 views

Hyperparameter tunning for MLP [closed]

I am trying to train a multilayer perceptron model and I need help deciding how to tune the hyperparameters. Any help/guide on how to choose the values of the hyperparameters? The outcome is binary ...
user avatar
1 vote
0 answers
17 views

Most Random Forest Features are the same

I have a random forest with let's say 256 trees and say 2000 features. from data processing to actual prediction, it takes several hundred microseconds, I was wondering if there were any ways I could ...
user avatar
0 votes
0 answers
5 views

I have troubles with getting a neural network to learn a function [duplicate]

I am currently trying to make an artificial neural network learn a function that takes a 1 dimensional input and the output is also of dimension 1. However, I have some troubles making the network ...
user avatar
0 votes
0 answers
13 views

How is the pdf of Dirichlet distribution defined at $x \in \mathbb R^K$ if $x_i = 0$ for some $i \in \{1, \ldots, K\}$?

I'm reading this slide about Dirichlet distribution. Generalizes Beta distribution to the K-dimensional simplex $\mathbb{S}^{K}$. $$ \mathbb{S}^{K}=\left\{x \in \mathbb{R}^{K}: \sum_{i=1}^{K} x_{i}=1,...
user avatar
  • 391
0 votes
0 answers
9 views

Sampling from a Posterior Dirichlet Process by knowing how to sample from a Dirichlet Process

A Dirichlet Process can be defined in the following way: If $a>0$ and $G$ a probability measure then the discrete probability measure of $\Theta$ which can be defined as $$\Theta = \sum c_{k}\...
user avatar
  • 1,679
0 votes
0 answers
3 views

Help with a generalized linear mixed effects model of a forestry trial

I am having troubles coming up with the proper statistical model and R code for a forestry trial we did several years ago. We looked at whether wood piles left over in clear cuts were beneficial for ...
user avatar
  • 1
0 votes
0 answers
13 views

Impact of L1 and L2 regularisation with cross-entropy loss

When we are dealing with Mean Square Error (MSE) loss function in optimization problems, we often add $L_1$ or $L_2$ penalty terms (or a combination of both) to the MSE loss function while training. ...
user avatar
1 vote
1 answer
9 views

Determine the effect on margins of a price increase

I hope you can help guide me in the right direction! Any advice is appreciated! Situation I'm currently analyzing the effect of a price increase from a retailer on a few 100 products. I'm interested ...
user avatar
0 votes
0 answers
5 views

Verifying token generation rate using generated tokens

Let's say I have a machine that makes tokens at certain rates. Tokens are generated every time someone pushes a button. Each button push can generate one of 4 different tokens, 45% token A, 36% token ...
user avatar
2 votes
1 answer
19 views

What test to compare two frequencies?

In my project I have to study the vigilance behaviour of birds, and I need to determine if adults start escaping from an approaching human more often than chicks. So I have the frequencies of adults ...
user avatar
  • 121
0 votes
0 answers
7 views

Optimal number of Clusters [which]

I'm trying to find out the optimal number of clusters through different means. First off I started with kmeans and after the graphic I obtained I thought 4 would be the most optimal number of clusters....
user avatar
0 votes
0 answers
5 views

Are there tools for pattern mining? [closed]

I have a set of data that I'd like to search for patterns. 10 columns of 25 rows, with 5 different mutually exclusive values in each cell. If no patterns are found, I'd like to explore the possibility ...
user avatar
  • 1
0 votes
2 answers
20 views

Is there name for statistical thinking of -- say a basket of 60% red, 40% black balls --- NOT guessing optimally?

I remember reading about this. Say there's a basket of 60% red balls and 40% black balls. And many statistical laymen if say given a reward for a correct guess -- they will guess red and black and etc....
user avatar
  • 243
0 votes
0 answers
6 views

What is the best way to find objects for which i have example images?

I have a bunch of images which contain objects which I want to detect in other images. The reference images are rectangular and each contains only one object which is presented in portrait mode. But ...
user avatar
  • 101
0 votes
0 answers
17 views

Write the PDF of an exponential prior given E[$\theta$] = 2

I am reviewing old exercise solutions and the following info is given: Assume that the conjugate prior for θ (as a special case of the gamma distribution) is following the exponential distribution ...
user avatar
  • 81
0 votes
0 answers
6 views

assessing impact of concurvity in GAM using time series of weather and mortality data

I have some questions about concurvity in a GAM using time series weather and mortality data (for details and model output see below). I'd be really appreciative of some help as most questions about ...
user avatar
  • 43
0 votes
0 answers
6 views

Meaningful to retrieve original value after standardization using clustering

I already referred these posts here and here. Currently, I am working on customer segmentation using their purchase data. So, my data has below info for each customer Based on the above linked posts ...
user avatar
  • 1,792
0 votes
0 answers
4 views

Quadratic causal mediating effects

Suppose we have one outcome variable $Y$, one treatment, and one mediator variables $M.$ We write the structural model as: $$ Y=\beta_{0}+\beta_{1}T+\beta_{2}T^{2}+\beta_{3}M+\epsilon $$ $$ M=\alpha+\...
user avatar
  • 735
1 vote
0 answers
8 views

How to derive solution to loss function of GLasso for precision matrix

I am trying to find the parameter $\hat\omega = min_{\omega}\Big(-log|\omega| + tr(S\omega) + \sum_{i,j}\lambda|\omega_{ij}|\Big)$ This is to regularize the precision matrix $\omega$ for the GLasso. ...
user avatar
-1 votes
0 answers
3 views

How to cluster binary data using excel or pandas dataframes? [closed]

I have a CSV with 101 rows and 150 columns. I need to find a way to segment/cluster the ID's using the Column values. It can be a machine learning approach or just using Excel techniques. At the end ...
user avatar
0 votes
0 answers
5 views

Proof of partial correlation formula [closed]

Given two regression: yt = αy+βywt +uy,t xt = αx +βxwt +ux,t Prove that ρx,y|w = cor(x, y)−cor(x,w)cor(y,w)/sqrt((1−cor(x,w)^2)sqrt((1−cor(y,w)^2)
user avatar
  • 1
0 votes
0 answers
8 views

How to generate confidence interval for population proportion for a multi class variable

I need to find the appropriate test for the following scenario: I have a population (let's say of an unknown size, for now) which I sampled from it and found the following distribution for some ...
user avatar
0 votes
0 answers
6 views

can i compute an interaction variable by multiplying a standardized variable and a mean centered variable?

I need to do a multiple hierarchical regression analysis for my master thesis. my continues independent variable is: online exposure to cannabis use. I used five items/questions/variables to compute a ...
user avatar
0 votes
0 answers
11 views

Interpreting error bars when conducting a one-sided test

I have learned that when interpreting error bars on a bar chart, given that the sample sizes are equal (or nearly equal), that we can trust the error bars if they overlap (or not) in terms of ...
user avatar
  • 1
0 votes
1 answer
30 views

What is the uncertainty of Leave-one-out-cross-validation method?

I have used the LOOCV to validate my model. As we know, the LOOCV method is a special case of cross-validation where the number of folds equals the number of instances in the data set. Thus, the ...
user avatar
1 vote
0 answers
8 views

How to simulate events in certain time?

I am trying to make simulation, and I have some events that happened durring a day. Some times it s for example few events durring half hour for one day, and sometimes we have event, and then few ...
user avatar
0 votes
0 answers
7 views

Statistical comparison of hierarchical data with pair-wise comparison

I am a bit confused how to use hypothesis testing here and please let me know if this question was already asked. Imagine the following hierarchical setup: There are 8 schools in a city. Within each ...
user avatar
0 votes
0 answers
11 views

ANCOVA - Follow up tests

Which follow up tests should be used with an ANCOVA to make sure the covariate is included in the assessment ? I’m struggling to find an appropriate test for this.
user avatar
  • 43
0 votes
0 answers
7 views

Is there a mathematical way to show why exactly adjustment for baseline helps in longitudinal randomized studies and NOT in non-RCTs?

How exactly adjustment for baseline helps to minimize the bias and increase effectiveness in randomized trials? I heard about: it minimizes the unexplained variance - but how? it minimizes the ...
user avatar
1 vote
0 answers
14 views

Under what circumstances is a m-class classification Neural Network equal to a base category logit model for nominal responses

Here is the setup for an m-class classification neural network. $$z_j=\sigma(\alpha_0j+\alpha_j^T x), (j=1,...,k)$$ $$t=\beta_0+\Sigma^k_{j=1}\beta_j z_j)=\beta_0+\beta^Tx, (l=1,...,m)$$ $$ y=f(x)=g(t)...
user avatar
0 votes
0 answers
18 views

Help with performing a time series analysis for Master Thesis (with R) [closed]

I was wondering if you guys had any advice on how to perform a time series analysis? For the context, I would like to analyze the link between projects related to video games on Kickstarter and the ...
user avatar
  • 1
0 votes
0 answers
6 views

Euclidean distance between points in PCA space along different principal component dimensions

I've picked up this project half way through, and I'm working through the last guy's code, so please bear with me. So the original data consists of 500+ points in 150 dimensions, and I want to ...
user avatar
0 votes
0 answers
5 views

How to incorporate a variance-covariance matrix from abnormal returns and using this to weight the returns in computing the test statistic, using R?

I want to run GLS model in my research, using R. My general specification is the following (cross-sectional regression): Abnormal ~ Country_dummy + Industry_dummy + ESG + Cash + STD + LTD + Size + ...
user avatar
  • 1

15 30 50 per page
1
2 3 4 5
3887