Questions tagged [computational-statistics]

Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.

226 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
614 views

How to set up a posterior predictive test quantities (Bayesian context) to check for independent Poisson distributions?

Suppose we are given data $y_j \sim \text{Poi}(\lambda)$ and assume $y_j$ are iid. We can assume the prior distribution for $\theta$ follows $\text{Gamma}(\alpha, \beta)$, The posterior ...
1
vote
0answers
278 views

Dixon test for outlier but which one is an outlier?

Following my previous question, I used Dixon test for outliers with the help of Michael Chernick answer. So now I have pvalues for say 10 numbers (basically 10 patients). But I have around 50k pvalues ...
1
vote
0answers
290 views

Computing the expected value using the sample mean gives poor convergence?

This is for Project Euler problem #371: http://projecteuler.net/problem=371 My solution was to generate a huge random pool of numbers between 0 and 999 inclusive, repeatedly ask the question 'How ...
1
vote
1answer
118 views

How to compare two data sets with similar standard deviation and different type

I know my title is a little bit weird but here's an example. I was working on a project about RPG game. To be more simple, there are four values: physical - attack & defence magical - attack &...
1
vote
3answers
758 views

naiveBayes does not give expected probabilities

I do not understand how the naiveBayes method from the e1071 package is calculating probabilities when classes are perfectly separable (more generally in categories where there are only one class ...
1
vote
1answer
226 views

What is SVM regression? Is it for regression or classification?

I'm trying to understand what is SVM regression. It's used for classification or regression? Can someone give an intuitive understanding of it?
0
votes
1answer
11 views

Unable to calculate kappa or weighted kappa when 1+ raters give the same rating to everyone — what is a good solution?

Are there alternate inter-rater reliability statistical tests one can perform in cases where a rater gives the same rating to everyone? SPSS is unable to return a kappa or weighted kappa value when ...
0
votes
0answers
13 views

Statistical Process Control - Variable Forgetting Factor in recursive PCA

Can someone explain to me how can the frogetting factor remain within the limits [0,1] with the definition of variable forgetting factor proposed by Lane et al (2003) for recursive PCA? If the ratio ...
0
votes
0answers
12 views

Calculate the sentiment knowing the single probabilities

I am working on finding the sentiment of some text. Right now I am using a python library called VADER that for each sentence it gives me back the probability that a specific phrase is positive, ...
0
votes
0answers
30 views

Is there a multivariate analogue of the Discrete Inverse Transform Sampling?

Context : I have been tasked for an assignment with the implementation of a Metropolis-Hastings algorithm to simulate the Ising model. In short, this model consists - in the case I'm considering - in ...
0
votes
0answers
13 views

comparing ratios between every sample across groups

I'm pretty new to statistics. I have some data sets: ...
0
votes
0answers
14 views

Calculation of MISE for density estimation in deamer package?

The mean integrated square error of a density estimate $\hat p$ of a true density $p$ is $$\begin{equation} \begin{aligned} \text{MISE}(\hat{p}) &\equiv \mathbb{E} \Big[ \int (\hat{p}(x,\mathbf{X}...
0
votes
0answers
13 views

How do you normalize results when dealing with two or three datasets?

I'm needing some help with a data analysis. I have taken advanced engineering calculus classes, but no advanced statistics, so I will need answers dumbed down a little, but hopefully I can understand ...
0
votes
0answers
9 views

What is a “filter” and what does “filtering” mean in statistics/engineering/computer science?

I see the term "filter" in many neuroscience papers with heavy statistical content ("spatial filter", "temporal filter", etc.), and sometimes little or no statistical ...
0
votes
0answers
11 views

Do I need to use the complex conjugate when convolving two functions with the FFT?

I know that, due to the convolution theorem, two densities $f$ and $g$ can be convolved by (i) applying the FFT to both of them, (ii) multiplying the results, (iii) applying an inverse FFT. Since I ...
0
votes
0answers
11 views

Quantify a score (or figure-of-merit) of the code style for source code of different sizes

I want to have a numeric score between 0 and 100 to reflect the quality of code style for the R source code written by my team members based on some pre-defined rules. The rules and checks are taken ...
0
votes
0answers
23 views

Statistical test for pair-wise comparisons of 36 different counts

I have 36 different population types that vary in combinations of three parameters (s, g, d). g and g has 3 discreet variations (0.2, 0.8, 0.95 and 10, 100, 500 respectively) and d has 4 (1.0, 0, 0.1, ...
0
votes
0answers
9 views

Normalization changes the effect of a variable in fixed effect logit

I have constructed a conditional/fixed effect logit model. My data consists of 6k+ groups in which I get two different coefficients for the same variable if I divide that variable to the the size of ...
0
votes
0answers
19 views

How to estimate the probability on an edge appearing in the minimum spanning tree of a graph?

I've been running into this problem recently and I've been stuck on it for a while. I have a set of vertices 𝐺 that form a complete graph. From this I need to sample 𝑘 vertices (which would also ...
0
votes
0answers
12 views

Numerically validating rates of convergence of approximations of expectations?

In applied mathematics it is standard practise to often validate theoretical approximations using numerical simulations. Since these simulations typically use numerical methods that convergence very ...
0
votes
0answers
47 views

Can anyone explain this adaptive kernel density estimation method?

I happened to need a reliable multivariete kernel density estimation method, and found it in this Matlab code, which turned out to work very well with my data. But I have trouble understanding its ...
0
votes
0answers
16 views

Stochastic gradient decent using local computation

I am new to the ML field especially the federated learning topics, and I am trying to use this formula to estimate local computation in FL. In this paper they provide the equation for local ...
0
votes
0answers
15 views

consistency between ranked lists

I have 5 features used in 3 models to estimate an outcome. Then I checked their significance in the 3 models to see which features are more significant. I got 3 lists of significant features, one for ...
0
votes
0answers
27 views

How to explain why categorical inputs (as is) versus categorical inputs (as binary) produce different coefficients in logistic regression

Background When using logistic regression, I observe that using categorical inputs (as-is i.e. leaving them as categorical with type factor) versus using said categorical inputs but transforming them ...
0
votes
0answers
17 views

Is my plot wrong? This is a plot of auc against lambda

I am trying to fit a logistic regression model using glmnet package. My data consists of 5 columns and 748 observations. I want to predict if blood donor will ...
0
votes
0answers
41 views

Computation of the density of the ratio of two random variables

Background: For two continuous random variables, $X$ and $Y$, the density of $Z := \frac{X}{Y}$ is given by \begin{equation} p_Z(z) = \int_{-\infty}^\infty \lvert y\rvert\, p_{XY}(zy, y) \, \text{...
0
votes
0answers
6 views

How can I transform evidence scores of single-video machine analysis output to confidence scores?

For my research purpose, I need to convert the evidence scores of machine vision output to confidence scores. I found below codes for R in the website: https://rdrr.io/github/bigpas/face_exp_analysis/...
0
votes
0answers
17 views

Lexical diversity of a text segment in comparison to the whole corpus/dataset

I've been reading articles on formulas to measure lexical diversity, but I haven't found yet a possible solution to my problem. I've been able to implement in C# a few formulas to measure lexical ...
0
votes
0answers
7 views

Number of Susceptible individuals in stochastic SIR ICM

I am using SIR-ICM to model an outbreak in large population (around 10 million). I want to know the size of susceptible individuals to use in this model. I am in a doubt to use the actual population ...
0
votes
0answers
17 views

How to handle time series data with multiple discrepancies?

I'm trying to classify time-series data where we have n hikers traveling between two locations. We're just trying to classify them without regards to seasonality The problem is that each hiker starts ...
0
votes
1answer
79 views

Difference Between Sensitivity Rate & Hit Rate

I've been told by my professor that hit rate is more important when it comes to real-life deployment but sensitivity and other accuracy rates need to be satisfactory also. However, my other professor ...
0
votes
0answers
36 views

Matrix formula for the correlation matrix, page 72 from Mathematical Tools for Applied Multivariate Analysis

Book information The book is titled: Mathematical Tools for Applied Multivariate Analysis By Paul E. Green and J. Douglas Carroll The print that I have is from 1976 The ISBN number is ...
0
votes
0answers
62 views

complexity of empirical estimator

Assume we have i.i.d. data $x_{1}, \dots, x_{n}$ from discrete distribution. Then, let's us consider empirical estimator: $$ \hat{p}_{i} = \frac{ \sum_{j=1}^{n}1(x_{j}=i)}{n} $$ What is the ...
0
votes
0answers
26 views

How to evaluate the ratio of (large) gamma functions in the normal inverse gamma marginal likelihood?

I am doing some Bayesian modeling where I am using Normal-Inverse-Gamma as prior for the unknown mean and variance, and Normal for likelihood. In my application, I need the marginal likelihood, where ...
0
votes
1answer
747 views

Determining shape parameter for Generalized Pareto Distribution Scipy

I have a set of values to which I want to fit a Generalized Pareto Distribution. Scipy provides functions for doing so: https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.stats....
0
votes
0answers
23 views

Identifying interaction terms in nonlinear data whose underlying function may be unknown

This is the data that I am using to frame and ask this question (code written in R): ...
0
votes
0answers
14 views

Do I need to calculate proposal probability using random walk on exponential space?

I know I can use symmetric random walk to avoid computing the proposal probability. However, I'm not sure if doing symmetric random walk on transformed space should also cancel the proposal ...
0
votes
1answer
44 views

Changepoint detection and confidence score

I am familiar with Bayes Theorem and hypothesis testing, but not much above that. However, I have a problem that I cannot seem to formulate in frequentist terms - to my knowledge. From what I know ...
0
votes
0answers
207 views

Ordinal multinomial logistic regression on one-hot encoded data

I have a task I am unable to tackle by principle. I'm working on survey data for one of our clients such that my design matrix is made of one-hot vectors with 15 features (originally 3 variables with ...
0
votes
0answers
128 views

Random-effects-meta-analysis-simulation: zero-estimates for tau^2

I am working on simulating a random-effects-model for comparison of the DerSimonian-Laird-method vs. Hartung-Knapp-Sidik-Jonkman-method in R. To do so, I chose different combinations of mu (true ...
0
votes
0answers
105 views

Must MSA be re-performed after annual instrument calibration?

I'm working in an automotive company that must follow IATF (International Automotive Task Force) requirements. I administrate a test-bench that performs radio frequency tests using an appropriate ...
0
votes
2answers
102 views

SPSS Statistics

The variance measures how far each number in the set is from the mean I have 151190 records of a table. and i have to analyze the data on SPSS. after the analysis i have some results. Mean=74617.92 ...
0
votes
0answers
477 views

Computing Steady Probabilities for Markov Process

I hope I am asking this question on the correct SE, if not, please redirect it. I am trying to compute steady state probabilities for a Markov process using python. Knowing the transition rate values ...
0
votes
1answer
117 views

Filtering out bursts with a consistent range from a time series

I have time series of bursts that look like this: … and zoomed in: Now, there are also spurious bursts (which I call noise) in the data, which look like this: … and zoomed in: As you can see, the ...
0
votes
0answers
135 views

Is this model really saturated? Is there a good alternative to an ANOVA for saturated models?

I have chemical data from groundwater wells that were exposed to two separate amendments of vegetable oil and monitored over several time points. I am trying to set up a 2-way ANOVA to test if the ...
0
votes
0answers
432 views

Comparing trends in 2 time series with different units, scales, directions of change?

I have two time series data as shown below in the figures. I want to compare if the rate of change is same in the both the line plots regardless of the direction of change (i.e increasing trend or ...
0
votes
0answers
19 views

Data Correlation for computing software metrics

I have the below sets of data per application, you can call them as software metrics. These metrics vary depending on the size of an application. ...
0
votes
1answer
84 views

Computing new variable of nominal and scale z scores variables

I'm analyzing a set of data where participants' reaction time and number of errors made during the task were measured. As there were two trials (where same data was measured) I was thinking I could ...
0
votes
1answer
33 views

Statistical methods for small samples

Can you please suggest some statistical methods for datasets with few observations (aprox 100 values) that I could research more? I know about bootstrapping but I was wondering what else there is. To ...
0
votes
1answer
150 views

what is the analytical test to run in case of 1 measure for three groups

I have this case where the data look like that Trial Person 1 person 2 1 4.7. 3.8 2 7.1. 6.3 3 5.4. 4.5 I want ...