All Questions
195,679
questions
0
votes
0
answers
2
views
How was the skewness formula made/derived?
The skewness formula that shows up when I Google it is:
I know what each of these values are individually, but I’m having a hard time seeing intuitively why this specific formula give a value for ...
- 13
0
votes
0
answers
3
views
Hypothesis testing for increase in sales
A company is planning to setup the option for self checkout (SCO) in all of its stores.
As a trial, they are looking to estimate the benefits of SCO in terms of increase in transaction volume by ...
- 51
1
vote
0
answers
6
views
Relationship between Survival Distributions and Log Linear Regression in Accelerated Failure Time models
The accelerated failure time model (AFT) can be expressed as:
$S_{1}(t) = S_{0}(\frac{t}{\gamma})$ where $S_{0}$ is a specified baseline survival distribution and $\frac{1}{\gamma}$ is the accelerant/...
- 11
0
votes
0
answers
15
views
Finding the chi-squared distribution of a squared difference of two independent normal variables
Given two independent random variables $X\sim N(0, \sigma), Y\sim N(0, \sigma)$, what is the distribution of a variable $Q = (X-Y)^2/4$ ? What would be the expected value and variance of $Q$?
- 1
0
votes
0
answers
6
views
Are these two events independent (Comey and Mccabe tax returns)? How would we tell?
The NYT reports:
The former F.B.I. director and his deputy, both of whom former
President Donald J. Trump wanted prosecuted, were selected for a rare
audit program that the tax agency says is random.
...
0
votes
0
answers
5
views
I have two different data sets, uneven in terms of number of species. I haven't found good recent examples of publications with 2 phylogenies
Thanks for reading me. I have two different data sets, uneven in terms of number of species. The first one has 20 plant DNA sequences, and the second set is a morphological matrix with 95 species (...
0
votes
0
answers
5
views
If partial regression coefficient is zero, then $Y$ is independent of $X_i$ conditional on all other regression variables
In a textbook Causal Inference in Statistics - A Primer (p. 81), it says
Given the regression equation $$y=r_{0}+r_{1} x_{1}+r_{2}x_{2}+\cdots+r_{n} x_{n}+\epsilon$$
if $r_{i}=0$, then $Y$ is ...
0
votes
0
answers
13
views
p-values of Spearman correlation test: Different reporting between R and Python [duplicate]
I am using scipy's spearmanr and R's cor.test with method=spearman to estimate the p-value of the spearman correlation test when applied to some data.
When I run the python code I get
when I do it R, ...
0
votes
0
answers
8
views
Correlation of groups in simulating data
I am simulating time series data. Each simulation consists of a group of time series with nodes at lets say 3 levels. To get the data to adhere to the structure I only generate the terminal units (...
0
votes
0
answers
5
views
How to sample for compliance when a portion of the population is already known to be non-compliant?
Say I wanted a representative sample to estimate the portion of a population that is compliant on "doing their taxes correctly". Normally, I'd do a random sample of that population to make ...
- 303
0
votes
0
answers
9
views
Probability that two specific people are selected for IRS audit within 4 year period
A recent NYTimes article states that
The minuscule chances of the two highest-ranking F.B.I. officials — who made some of the most politically consequential law enforcement decisions in a generation —...
- 3,778
0
votes
0
answers
9
views
How to compare observed versus expected ratio data between two independent, unpaired groups with different sample sizes?
so I'm comparing the expected fitness of double mutant animals (based on independent, additive effects of each corresponding single mutant) to the observed fitness of the double mutant. The data is ...
4
votes
1
answer
23
views
What is wrong with treating everything as a hyperparameter?
I've seen a number of questions like this asking whether certain parameters can be treated as a hyperparameter. Why can't we just treat everything as a hyperparameter? I understand that this is an ...
- 141
0
votes
0
answers
7
views
Sample size estimation for difference-indifference design with continuous outcome
I have spent some time looking for a package in R to estimate the required sample size for a difference-in-difference design with continuous outcome. However, I could not find any package. Is there a ...
- 1
4
votes
1
answer
21
views
Distribution of sample variance of independent but not identically distributed normals
When I am reading the Wikipedia page on the chi-squared distribution, it states that if $X_1, \ldots, X_n$ are $\text{N}(\mu, \sigma^2)$, then $\sum^n_{i=1} (X_i - \bar{X})^2 \sim \sigma^2 \chi^2_{n-1}...
- 41
0
votes
0
answers
6
views
Normality test vs Gauss Markov assumption for panel data
I am doing fixed effect regression after conducting hausmann test on panel data.
I received significant results in line with what's expected for my model. My data set has around 6000 observations and ...
2
votes
0
answers
26
views
Does Central Limit Theorem (CLT) apply to Regression Coefficients?
The way I learned about Central Limit Theorem in school is illustrated in the following example:
Suppose you have a population of 100,000 basketball players. You are interested in knowing the average ...
- 5,922
0
votes
0
answers
4
views
Regression discontinuity with stratified assignment to treatment
I am working on running a regression discontinuity model where the "random" assignment to treatment occurs at a different level from the outcome variable.
Essentially, the forcing variable ...
- 1
0
votes
0
answers
6
views
How is the VAE encoder and decoder "probabilistic"?
In a VAE, my understanding is that the encoder takes in $x$, outputs a vector $(\mu, \sigma)$ that characterizes a certain normal distribution $q(z|x)$. Then we sample from this distribution to get a ...
- 153
2
votes
0
answers
10
views
Why is the ICC near 0 in this longitudinal data?
I’m working with a toy data set to get a basic understanding of longitudinal data analysis and I’m having a difficult time with a conceptual understanding of the intraclass correlation (ICC) as a ...
- 21
0
votes
0
answers
11
views
How to do inference between empirical CDF and unknown CDF using Wasserstein distance？
I try to find a simple example of using Wasserstein distance for carrying out the goodness-of-fit test. As in the paper: https://www.gwern.net/docs/statistics/probability/2019-panaretos.pdf. In the ...
- 294
4
votes
1
answer
28
views
Can there be functions $g$ and $f$ such that $\rho_{f(X),g(X+Y)}^2 > \rho_{X,X+Y}^2$
Motivation a special case
A special case of this question is an inequality between the Spearman's rank correlation and the Pearson correlation (Why is the sum of individual Spearman's rho squared ...
- 48.8k
1
vote
0
answers
8
views
How can I find Conditional Probabilties from dataset points of features (random variables)?
I am trying to solve the following at work and will dummify for the sake of making it easier to explain myself and getting an answer. My main query is about Step 4 below. But if something is wrong or ...
- 422
0
votes
0
answers
13
views
Reasonable to incorporate sample size into beta-binomial?
Setup:
The relationship between the beta and binomial distributions is well known.
$$\frac{\pi^{\alpha - 1} (1 - \pi)^{\beta - 1}}{B(\alpha, \beta)} \leftrightarrow {{n}\choose{x}}\pi^{x} (1 - \pi)^{n-...
- 520
2
votes
1
answer
14
views
How can I calculate the optimal formula for predicting an outcome based on a lab value?
How can I calculate the optimal formula for predicting an outcome based on a lab value?
e.g. a rise in liver function tests can be predictive of liver disease.
Having liver disease (or not) is the ...
0
votes
0
answers
6
views
What is the difference between the latent variables $z$ and the unknown parameters $\theta$ in EM algorithms?
I am reading about variational inference and EM algorithms. In reading about variational inference, the model
\begin{aligned}
y &\sim \mathcal{N}(\beta x, \sigma^2) \\[.5em]
\beta &\sim \...
- 243
5
votes
0
answers
31
views
Are mixed models necessary if random effects estimates are close to zero?
I am currently using (general) linear mixed models as a mean to avoid pseudoreplication and control for measures made on various individuals in the same location (transect). I'm now starting to think ...
0
votes
0
answers
4
views
Estimate standard error due to reduction in sampling rate for a time series data
I have a large set of time series data representing the progression of a variable during an industrial production process collecting at a certain frequency at regular intervals. The mean of each curve ...
3
votes
1
answer
38
views
Parameter estimation of state-space models with hidden variables
I have a time-series analysis problem, that I am having trouble finding a suitable regression technique for.
I have a coupled linear three dimensional system
\begin{align*}
X_{t} & =\left(1+J\...
- 31
0
votes
0
answers
8
views
Why does the formula of IDF in TD-IDF technique use log? [duplicate]
I understand that the idea of IDF is to measure the importance of a term across a corpus by weighing down the terms that are very common across a corpus and weighing up the rare ones.
Formula:
$$IDF(...
- 111
0
votes
0
answers
20
views
Can the multivariate-t distribution have density greater than 1? [duplicate]
I'm working with a Python implementation of the Multivariate T distribution, and I've noticed when I evaluate the PDF at certain points, the likelihood returned is > 1. This is causing issues in ...
- 101
0
votes
0
answers
20
views
How to ensure independence and equality of variance for errors in neural networks?
In an artificial neural network, defined as,
\begin{equation}
\boldsymbol{z}_{\ell}=\boldsymbol{y}_{\ell-1}\boldsymbol{W}_{\ell-1,\ell}^T,
\label{eq:z}
\end{equation}
\begin{equation}
\...
- 643
0
votes
0
answers
13
views
How is bootstrapping used in machine learning? [duplicate]
I understood bootstrapping in the statistical context.
Example: we have a sample of 1000 people. We want to know their mean. We pick 5 people at random (with replacement) for 20 times and we compute ...
- 133
0
votes
0
answers
6
views
How do you count the number of parameters in a mixed model? [duplicate]
I'm trying to build my first linear mixed model. I ran some experiments in which I looked at whether the biomass of 40 ecosystems depended upon two conditions (x1 and x2) and their interaction (x1*x2)....
- 21
0
votes
0
answers
7
views
rlmer error message
I'm trying to run an rlmer model which uses the same random-effects structure as the most parsimonious model derived from a previous simplification procedure with lmer models. The model structure is ...
- 1
0
votes
1
answer
41
views
How to avoid estimating prices that are more than 25 dollars off of the actual price in Machine Learning model?
I am currently working on a case study where I have to estimate how much a person makes by giving their property for rent. They provided me with a constraint which is as follows:
"avoid ...
- 109
1
vote
1
answer
12
views
Using an ARIMA model to output many different scenarios of future wind generation
I am looking to model potential scenarios of wind generation for next year (specifically August). I have read through the literature and decided on using an ARIMA model. I have 10 different data sets ...
- 13
0
votes
0
answers
10
views
How to select a threshold when retraining on the whole dataset?
I am training a Resnet model for a binary classification task, and since my dataset is quite small, I'd like to estimate my performance by doing a 10-fold cross validation, then retrain on the whole ...
0
votes
0
answers
8
views
GAM with interactions: is my model natural?
I'm trying to model the distance of a projectile using a GAM. In particular, I have covariates Exit Speed (initial exit speed of the projectile), Angle (angle at which the projectile was launched), ...
- 169
0
votes
0
answers
11
views
Approximate p-values for Relative Risk Reduction
I am working on a problem in which I compare the probability of infection within a region before treatment, and again after 5 years of Mass Drug Administration (MDA). Below describes the sampling ...
- 520
0
votes
0
answers
18
views
What can we say about $\log \zeta(\mathbf X)?$
I previously asked: Visualisation of L(X). Unfortunately I didn't receive an answer. This question is related to that question with some new elements...
Let $\mathcal L^n_+$ the set of all $n$-...
- 111
0
votes
0
answers
19
views
Understanding Zscore using coin toss
I am working on a coin toss exercise where the coin is biased (let's say 60% heads) and I have been asked 3 different questions:
What is the optimal betting strategy in that situation ?
Understand ...
- 11
2
votes
1
answer
23
views
No Variance in Monte Carlo Simulation
I wanted to do a Monte Carlo simulation for some of the electoral districts in my state in the upcoming US midterms. My methodology essentially was as follows:
I have a list of populations and vote ...
- 123
0
votes
0
answers
6
views
Significant Levene-Test in Mixed ANOVA
I conducted a study to investigate the effect of instagram profiles on mood.
Therefore I created two profiles on instagram and used a pre-post-measurement of mood (PANAS-scale). Participants (N=130) ...
- 1
0
votes
0
answers
13
views
Confidence Interval for the Mean of Sample Means
This may seem like a very stupid question, but I'm just wondering what is the most appropriate way to calculate it.
I have two sample means $\overline{X}$ and $\overline{Y}$ that come from normally ...
1
vote
0
answers
17
views
Is there a formal methodology for controlling a variable in a tree based model?
I am currently building a xgboost model to classify data into 6 categories of risk for insurance policies. I have 5 years worth of policy holder data, including policy year. When building a GLM for a ...
- 11
0
votes
0
answers
6
views
comparison of standardised residuals from chi-square goodness of fit test
I am analysing the observed distribution of certain amino acid sequence motifs across the superkingdoms against that predicted based on theory using chi-square goodness-of-fit test. In order to find ...
- 11
0
votes
0
answers
19
views
How to decide which variables to smooth in GAM
When specifying it's formula GAM has s function for smoothing.
Let's say I want to fit mtcars.
I can do the following:
Approach 1:
...
- 101
0
votes
0
answers
7
views
Expected number of random cell visits in Q-learning
I'm interested in Q-learning with a time-decaying exploration rate
$$ \varepsilon_t = \exp(-\beta t)$$
In order to evaluate how `reasonable' certain values of $\beta$ are, and to make them comparable ...
- 925
1
vote
0
answers
14
views
Should I use Chi-Squared or Fisher's test?
everyone. I have a question which involves statistics and the R programming language, thus I believe that this question pertains to this forum, but if any of you know someplace else where I could go ...
