All Questions
216,404
questions
3
votes
1
answer
37
views
Sampling line segments within a box
I'm at a bit of a loss for where to start with a sampling problem I'm having so any and all direction would be helpful. I essentially want to sample line segments of identical length within a bounding ...
- 31
0
votes
0
answers
6
views
Computing Expectations in Fisher Information Matrix for Gaussian Mixture Models
Consider a Gaussian Mixture Model (GMM) with $K$ components:
$p(x) = \sum_{k=1}^K \pi_k \mathcal{N}(x; \mu_k, \Sigma_k)$
where $\pi_k$ are mixing coefficients, $\mu_k$ are mean vectors, and $\Sigma_k$ ...
- 133
2
votes
3
answers
91
views
Low $R^2$ puzzle
I conducted an experiment and using the data I fitted a basic linear regression model. The top chart displays the fit. The $R^2$ is abysmally low indicating almost no explanatory power (adjusted $R^2$ ...
- 4,675
7
votes
6
answers
2k
views
In roulette, is the frequency of getting long sequences of reds lower than that of shorter sequences?
In a roulette game that continues indefinitely, is it correct to say that achieving a sequence of 100 consecutive reds will occur far less frequently (perhaps once in every million spins) compared to ...
- 73
0
votes
0
answers
20
views
What's the point of the left Fcrit value in a 2-tailed F-test?
In a 2-tailed F-test, there are 2 critical values right? One on the right and one on the left.
You can get the one on the right by using an F-table and the one on the left by getting the reciprocal of ...
1
vote
1
answer
29
views
Generate Quasi Random Numbers for a Multivariate DIstribution
Algorithms like Sobol or Holton provide quasi random numbers (that is, the numbers "look" random in the sense of a uniform distribution, but they are deterministic) in the hypercube $[0,1]^d$...
- 138
3
votes
1
answer
233
views
When trying to find the quartiles for discrete data, do we round to the nearest whole number?
I have 2 cases:
First, I want to find the first quartile to the set data: 1,2,3,4,5.
Normally we calculate the qaurtile as:
Now $Q_1 = \frac{2+1}{2} = 1.5$.
But they never state if its continuous (i.e....
- 131
1
vote
0
answers
18
views
Lower Tail Bound for Noncentral chi-squared distribution
A noncentral chi-squared random variable $Z$ is of the form
$$
Z = \sum_{i=1}^k X_i^2
$$
where $(X_1, X_2, \ldots, X_i, \ldots, X_k)$ be $k$ are independent, normally distributed with mean $\mu_i$ and ...
- 11
0
votes
0
answers
11
views
How to compute mutual information between a scalar random variable and a vector random variable? [duplicate]
I have a vector random variable $X = (x_1, x_2... x_n)$ where each $x_i$ has a discrete value. I also have a discrete scalar random variable $Y = y$. Is mutual information between the two defined? If ...
- 81
0
votes
0
answers
10
views
How to formulate this entity resolution problem?
Problem formulation
Given:
Two sets of elements: S = {s₁, s₂, ..., sₙ} and T = {t₁, t₂, ..., tₘ} (you can think of them as two sets of records)
Each element e ∈ S ∪ T has properties:
A real number v(...
- 1
0
votes
1
answer
30
views
A/B Test Proportions when one of the samples has zero responses
Hi everybody I made an experiment about the conversion of customers that had some responses in group A but 0 responses on group B. If I input these values in a two-sample proportions_ztest It gives me ...
0
votes
0
answers
20
views
Should I use ROC curve in my training set after training a Random Forest classification model with k-fold cross validation?
I have a conceptual question: after dividing a dataset into a training and test set (70:30), both are balanced and shuffled, should I use the Confusion Matrix and the ROC curve of a model generated by ...
2
votes
1
answer
74
views
Derivation of expectation with subscript
I am going through the derivation for Denoising Diffusion Probabilistic Models (DDPMs) based on Calvin Luo's Diffusion tutorial, where he finally develops the reconstruction term, the prior matching ...
0
votes
0
answers
24
views
A Method for Functional Misspecification
I have been thinking about unbiasedness recently. An idea for came to my mind . Imagine that the dataset is separated into three different parts as training, the data that will be added to training ...
0
votes
0
answers
18
views
Suitable function of BLUPs in lme4 package [closed]
I want to analyze a data set includes 12 genotypes across 2 conditions or environment(drought stress and normal) in 4 years.
I used the analysis of an article that used the BLUPs method as my analysis ...
0
votes
0
answers
7
views
training convergence stagnation in image reconstruction using Vgg19 [closed]
--------------------------------------Context---------------------------
I am trying to implement this paper Universal Style Transfer via Feature Transforms. In the paper they trained 5 decoders for ...
0
votes
0
answers
5
views
Why is parallel bootstrapping in R slower than sequential bootstrapping (using boot and future packages)? [migrated]
I'm conducting a bootstrap analysis using the boot package in R with the option parallel = 'snow' (for Windows computer). However, computing in parallel is slower ...
- 3
1
vote
0
answers
29
views
Motivation behind this exercise problem on complete sufficient statistic
This is from Hogg and McKean's "Introduction to Mathematical Statistics" Chapter 7 (Sufficiency), section 7.4 (Completeness and Uniqueness).
Exercise 7.4.10.
Let $Y_1 < Y_2 < \cdots &...
3
votes
1
answer
70
views
Correctly modelling a continuous log dose-response relationship in meta-regression for small dosages
I’m looking at the effectiveness of a type of programme that can have a range of number of sessions, 1 to 20. I start with a meta-analysis (k = 70) that tells me the average effect size for that type ...
- 43
1
vote
0
answers
27
views
multilevel regression, model specification
We would very much appreciate your advice on what’s the best way of analysing our study design:
We studied the effect of media of delivery (i.e. augmented reality vs video) of educational content on ...
- 111
0
votes
0
answers
37
views
How to evaluate estimation of treatment effect under wrong linear model ignoring replicated measurements?
A Simulation study with the following task is given:
Do 1000 simulation runs based on simulated data from the model below to explore the following issue:
How well is the coefficient of $x_1$ (...
- 75
2
votes
0
answers
37
views
Standard deviation of predicted value of a regression equation
In a typical regression setup with residuals are independent, the standard deviation of the new value of response variable would have been
${\sigma} \sqrt{ \left(1 + a^T { \left(X^T X \right)}^{-1} a\...
- 41
1
vote
1
answer
13
views
Title: Issues with Fitting a Quasi-Periodic Function with a Trend Term in Python
I'm trying to fit a quasi-periodic function with a trend term to my time series data using Python. The data consists of monthly observations from 1963 to 1976. My goal is to model the data with a ...
- 369
2
votes
1
answer
23
views
as.POSIXct results in NAs for some of my data
I want to convert my date column into
Reproducible dataset:
...
0
votes
1
answer
42
views
Drawing uniformly from a skewed distribution
Are there methods to draw samples from a skewed population in such a way that the samples are uniformly distributed?
For example, if I want to show a correlation between height and salary, but the ...
- 83
0
votes
1
answer
88
views
Difference between anova () and summary ()? [closed]
Original post summary:
I want to run a two-way repeated measures ANOVA.
What is the difference between the following functions: anova(), ...
1
vote
0
answers
22
views
Model still overfits after hyperparameter tuning, dataset balancing and convolution layering
I am trying to classify either an image of 25x25 px stacked together as 50x25 px is the same(1) or different(0). I am using keras to create the NN layers. There are 10,000 instances of both 1s and 0s ...
- 111
0
votes
1
answer
34
views
If you had a disease with a 50% chance of living 5 years and a 30% chance of living 10 years, at 5 years what odds now of getting to 10 yr [closed]
See title for all info needed. Please also provide the method/ maths.
- 1
0
votes
1
answer
38
views
Simulation of type-1 error rate, and how to check type-2-error (rate)
With $H_{0}:\mu = 0$,
iter = 100
n = 500
# alpha, beta
a = .05 # b(0.5) = .95 (given in the question)
and $x{\sim}\mathcal{N}(0.5, \sigma)$, for arbitrary $\sigma{&...
- 105
0
votes
2
answers
47
views
Question about regression [duplicate]
Let's assume that I have large samples of data for the price and quantity of a product. I regress this data and I find that the correlation is one. Since the residuals in a linear regression are 0 ...
0
votes
0
answers
10
views
Recursive Random Search and Categorical Cost Functions
I'm currently working on a project that involves optimizing the default Spark-submit configurations to minimize execution time. I've developed two models to aid in this process:
Binary Classification ...
- 155
0
votes
0
answers
4
views
Need Guidance to Develop a Recommendation System for an Edu-HR Tech project
TLDR: Looking for guidance on how to transform a user's problem statement into a machine learning problem and how to prepare myself to solve it.
Greetings, everyone!
I am currently a web developer ...
0
votes
0
answers
20
views
Alternative to random intercept cross-lagged panel model
I'm looking for some inspiration as how to best model a given phenomenon, thought I'd ask around here.
This is a study where we will be collecting 5 repeated measures (say blood pressure, BP) over the ...
12
votes
5
answers
2k
views
In what instances are 3-D charts appropriate?
All the data analysis-related texts I have read over the years recommend against using three-dimensional charts in almost all cases. To quote one of them, "Never use a 3-D chart when a two-...
- 221
0
votes
0
answers
26
views
Jeffreys' test for a single event
Assume $x\sim B(p,n)$ and using Jeffreys' prior the posterior distribution of $p$ for $x$ events and $n$ observations is $$\text{Beta}(x+\frac{1}{2},n-x+\frac{1}{2})$$ I am reading about a Jeffreys' ...
- 35
0
votes
0
answers
7
views
Panel time series methods and endogeneity
Do panel time series methods such as Pooled Mean Group Estimator rely on an exogeneity assumptions of the explanatory variables or does the presence of a cointegrating relationship guarantee that ...
- 363
2
votes
0
answers
48
views
How to evaluate mixed model using lmer(), anova() and bootstrap when omitting random slope?
A Simulation study with the following task is given:
Do 1000 simulation runs based on simulated data from the model below to explore the following issue:
Fit a mixed model with random intercept but ...
- 75
3
votes
1
answer
49
views
Can you use the beta-binomial distribution instead of MCMC?
So, I have a project to test the hypothesis that a marketing campaign with a new art generates more purchases than the old one, I have 2 samples of data, one using the standard ad and one using the ...
- 33
0
votes
0
answers
26
views
How to interpret type-1-error, when omitting fixed effect from mixed model?
A Simulation study with the following task is given:
Do 1000 simulation runs based on simulated data from the model below to explore the following issue:
Do simulations where the regression ...
- 75
0
votes
0
answers
33
views
+50
Stochastic blockmodel with continuous weights in [0,1]
I want to use a stochastic blockmodel (SBM) to cluster the nodes in a network where the edge weights are continuous numbers between 0 and 1. It seems that R-packages such as ...
- 26.4k
5
votes
1
answer
188
views
Is the matrix formed by taking the absolute values of the elements of a positive-definite covariance matrix still positive definite?
Suppose I have a positive-definite covariance matrix $\boldsymbol{\Sigma}$. I construct a new matrix $\boldsymbol{\Lambda}$, where each element $(i,j)$ in $\boldsymbol{\Lambda}$ is equal to the ...
- 1,310
0
votes
0
answers
5
views
Is the significance test in GMM more reliable than it in ML, especially when the regularity conditions are not met?
Maximum likelihood vs generalized method of moments
From the answer to the above question, we can draw a conclusion that GMM and ML can get the same parameters' estimators when the model is exactly ...
- 1
2
votes
1
answer
27
views
Testing for significance where results are expressed as a ratio to an in group control
I originally asked this question in the biology stack exchange site, but realise that it might be more relevant here in the stats site:
I have a large dataset that represents a library of yeast ...
0
votes
0
answers
22
views
+50
Literature for self-excited Markovian processes with power-law distributions
I am working on a model for time series of events with the power-law distribution of inter-event intervals. I went for a point process, governed by a stochastic differential equation
$$
d\lambda = -a \...
- 51
6
votes
1
answer
260
views
Percentage changes versus absolute changes when comparing rankings
We are analyzing the ranking changes of countries based on an economic index to determine whether these variations are significant. For example, Country A moves from rank 5 to rank 3, representing a ...
- 81
3
votes
0
answers
20
views
Validate if the data points are from a given distribution
I have a KDE object which is computed using 5000 training data points (3-dimensional). The KDE is computed using Statsmodel.
I have 35 test data points (also 3 dimensional).
Data for MWE:
Training ...
- 131
0
votes
0
answers
17
views
Dynamic factor analysis/Kalman filter using fixed scores/state and dynamic loadings/observation model?
I have to following data setting. There's $n$ genomes whose expression for a trait of interest is measured in $p$ locations. This is repeated for $\tau$ years so the total number of datapoints is $n\...
- 1
0
votes
0
answers
29
views
Simple but important question: how do you write down the formula for the probability density of data in general? [duplicate]
In machine learning many data can be thought of as generated from a probability density function (also called probability distribution).
But most probability textbook only discuss probability density ...
- 691
0
votes
1
answer
26
views
How to derive points for each variable in nomogram by hand?
I'm dealing with nomograms at this time, but I don't really understand how to derive the exact points for each variable by hand (or in a mathematically way).
To be concrete, I brought a simple example:...
unicorn
3
votes
1
answer
86
views
How do I test if a population change is statistically significant?
I have data showing different statistics for all parishes in the country for 2014 and 2024 (I actually have data for all the years in between as well, but I am comparing these two years). The data ...
- 33