Questions tagged [computational-statistics]
Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.
618
questions
-1
votes
0answers
13 views
Integral using importance sampling formula [closed]
I am trying to do an integral via importance sampling formula. The algorithm is as follows:
The value for the integral should give: 0.838932960013382. Also I must use the next $p\left( x \right) = \...
1
vote
0answers
26 views
What is this probability distribution?
Thank you in advance for any suggestions or feedback.
I have a discrete 1D probability distribution represented as a vector $\textbf{p}$, $p_i = p(x_i)$.
I am interested in finding the Wasserstein (...
1
vote
1answer
33 views
Choosing the appropriate statistical test- please help
This seems really simple, but I need some help choosing a statistical test. I conducted an experiment where I was trying to figure out what food 20 crickets preferred, potatoes or sugar. I put the ...
0
votes
0answers
24 views
Is there a multivariate analogue of the Discrete Inverse Transform Sampling?
Context : I have been tasked for an assignment with the implementation of a Metropolis-Hastings algorithm to simulate the Ising model. In short, this model consists - in the case I'm considering - in ...
0
votes
0answers
10 views
How large should be a bootstrapping sample? [duplicate]
The technique of bootstrapping, a technique used to describe estimators using sample data, is described by Greene (Econometric Analysis, 8th ed.) as it follows:
In such technique, sample size ($m$), ...
0
votes
0answers
12 views
comparing ratios between every sample across groups
I'm pretty new to statistics.
I have some data sets:
...
0
votes
0answers
9 views
Calculation of MISE for density estimation in deamer package?
The mean integrated square error of a density estimate $\hat p$ of a true density $p$ is
$$\begin{equation} \begin{aligned}
\text{MISE}(\hat{p})
&\equiv \mathbb{E} \Big[ \int (\hat{p}(x,\mathbf{X}...
1
vote
1answer
8 views
Shortcut for computing RSS at different split point when building regression trees?
I'm coding regression trees from scratch in R. For a given ordered predictor variable, $X$, obviously I have to compute the RSS at each unique ordered value of $X_i$. When moving over to the next ...
0
votes
0answers
11 views
How do you normalize results when dealing with two or three datasets?
I'm needing some help with a data analysis. I have taken advanced engineering calculus classes, but no advanced statistics, so I will need answers dumbed down a little, but hopefully I can understand ...
0
votes
0answers
9 views
What is a “filter” and what does “filtering” mean in statistics/engineering/computer science?
I see the term "filter" in many neuroscience papers with heavy statistical content ("spatial filter", "temporal filter", etc.), and sometimes little or no statistical ...
0
votes
0answers
9 views
Do I need to use the complex conjugate when convolving two functions with the FFT?
I know that, due to the convolution theorem, two densities $f$ and $g$ can be convolved by (i) applying the FFT to both of them, (ii) multiplying the results, (iii) applying an inverse FFT.
Since I ...
2
votes
1answer
40 views
Square root transformation of Poisson process. How $\small Var[\sqrt{P(\lambda)}] \approx \frac{1}{4}$
I am working on Kaggle Neural data challenge. I am trying to understand the transformation applied on the neural spiking data. A number of spikes given a stimulus are Poisson distributed as
$$Y_i \...
31
votes
14answers
2k views
If R were reprogrammed from scratch today, what changes would be most useful to the statistics community? [closed]
Many people in the statistics community and other academic fields use R as their primary language for data analysis and statistical computing. It is a wonderful ...
0
votes
0answers
11 views
Quantify a score (or figure-of-merit) of the code style for source code of different sizes
I want to have a numeric score between 0 and 100 to reflect the quality of code style for the R source code written by my team members based on some pre-defined rules.
The rules and checks are taken ...
0
votes
0answers
21 views
Statistical test for pair-wise comparisons of 36 different counts
I have 36 different population types that vary in combinations of three parameters (s, g, d). g and g has 3 discreet variations (0.2, 0.8, 0.95 and 10, 100, 500 respectively) and d has 4 (1.0, 0, 0.1, ...
0
votes
0answers
8 views
Normalization changes the effect of a variable in fixed effect logit
I have constructed a conditional/fixed effect logit model. My data consists of 6k+ groups in which I get two different coefficients for the same variable if I divide that variable to the the size of ...
1
vote
0answers
14 views
Generate data for significance testing
I want to generate a data set with a pre-specified significance level.
Let's say we have 2 covariates x1, x2, and an outcome variable y.
We fit a linear regression model as follow:
...
1
vote
0answers
60 views
How to computationally optimize ARIMA walk-forward validation?
Just fitted and ARIMA model
model = ARIMA(Preco1, order=(6,1,2))
model_fit = model.fit(disp=0)
In order to do the walk-forward validation of an ARIMA model on a ...
0
votes
0answers
14 views
How to estimate the probability on an edge appearing in the minimum spanning tree of a graph?
I've been running into this problem recently and I've been stuck on it for a while.
I have a set of vertices 𝐺 that form a complete graph. From this I need to sample 𝑘 vertices (which would also ...
0
votes
0answers
11 views
Numerically validating rates of convergence of approximations of expectations?
In applied mathematics it is standard practise to often validate theoretical approximations using numerical simulations. Since these simulations typically use numerical methods that convergence very ...
1
vote
0answers
95 views
How to create a Bayesian network? [closed]
I have the following question and I really need help please:
Consider the following hypothetical scenario. A car company would like
to use a Bayesian Network model to better predict whether a certain
...
1
vote
1answer
42 views
Fast Evaluation of a Double Sum
Let
$q$ be a probability distribution on $\mathcal{X}$,
$w$ be a nonnegative function from $\mathcal{X}$ to $\mathbf{R}$ which is bounded away from $0$ and $\infty$, and
$s$ be a bounded function ...
1
vote
0answers
39 views
How to model financial HFT time-series data with multi scale autocorrelation
I work with tick level time-series univariate prices data. Tick level means that there are hundreds to thousands observations per second. The observations are timestamped, so one can use both wall ...
2
votes
0answers
40 views
Computationally estimate $E[f(\hat \beta_1 X)]$ where $\hat \beta_1$ is the estimated coefficient obtained by ordinary least squares regression?
Let $(X_1,Y_1),(X_2,Y_2),\dots,(X_5,Y_5)$ be i.i.d samples and consider the regression model
$$
Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, \quad \quad \text{for} \ i \in \{1,2,\dots,5\},
$$
where $\...
2
votes
1answer
19 views
Can you use ANOVA to compare two models that are not nested?
Is it possible to use ANOVA to compare to non-nested models that have the same response variable? If not, what is the best way to compare non-nested models that have the same response variable?
0
votes
0answers
46 views
Can anyone explain this adaptive kernel density estimation method?
I happened to need a reliable multivariete kernel density estimation method, and found it in this Matlab code, which turned out to work very well with my data. But I have trouble understanding its ...
0
votes
0answers
16 views
Stochastic gradient decent using local computation
I am new to the ML field especially the federated learning topics, and I am trying to use this formula to estimate local computation in FL. In this paper they provide the equation for local ...
1
vote
0answers
25 views
Computation efficient time series causal graph representation?
I am trying to select the best features to train a Deep Learning model from a big database, consisting of stock/cryptocurrencies prices (and possibly technical indicators).
I have found this algorithm,...
2
votes
1answer
67 views
Is a pseudo-random number generator suitable for assigning treatments in an experiment?
I am working on a web service for randomly allocating users to treatment groups.
My belief is that using a pseudo-random number generator as the basis of those allocations is reasonable, but I am not ...
0
votes
0answers
12 views
consistency between ranked lists
I have 5 features used in 3 models to estimate an outcome.
Then I checked their significance in the 3 models to see which features are more significant.
I got 3 lists of significant features, one for ...
0
votes
0answers
26 views
How to explain why categorical inputs (as is) versus categorical inputs (as binary) produce different coefficients in logistic regression
Background
When using logistic regression, I observe that using categorical inputs (as-is i.e. leaving them as categorical with type factor) versus using said categorical inputs but transforming them ...
1
vote
2answers
72 views
Estimating error when bootstrapping results are skewed
I'm trying to figure out the most robust way to estimate the error on the best-fit parameters of an exponential model ($y = x^a$) for some data. I have heard that bootstrapping is a solid way to ...
0
votes
0answers
14 views
Is my plot wrong? This is a plot of auc against lambda
I am trying to fit a logistic regression model using glmnet package. My data consists of 5 columns and 748 observations. I want to predict if blood donor will ...
1
vote
0answers
17 views
Help with linear model fitting (MATLAB)
3:
Hello
I have fitted two separate linear models to a set of data (resit_eggs_data.txt), of which one spans 1950 to 1990 and the other 1990 onward.
I need help understanding question 4(a).
4(a): ...
0
votes
0answers
35 views
Computation of the density of the ratio of two random variables
Background:
For two continuous random variables, $X$ and $Y$, the density of $Z := \frac{X}{Y}$ is given by
\begin{equation}
p_Z(z) = \int_{-\infty}^\infty \lvert y\rvert\, p_{XY}(zy, y) \, \text{...
0
votes
0answers
31 views
Log Sum Exponential Trick On Weibull Mixture [duplicate]
I am trying to evaluate the log-likelihood of a mixture of weibull distributions and am running into problems with the numerical aspect. In short, I have $M$ mixtures and want to evaluate:
$$
\log \...
0
votes
0answers
11 views
Why maximum likelihood estimation is different in a logistic regression classifier? [duplicate]
why don't we use the simple linear regression method such as mean square for parameter fitting in logistic regression?
1
vote
0answers
13 views
Question on method for analysis
First of all, I have begun researching as a masters student and I am forcing myself on more statistical validation and significance testing - this is not my strong suit. As such, I am applying ...
3
votes
2answers
75 views
Is my logistic regression model correct?
I have a factorial design 2*2 (A and B). Both variables with two responses high (coded as 1) and low (coded as 0) and I have a response variable $y$, my logistic model include interaction between A ...
9
votes
6answers
997 views
Books Similar to Introduction to Statistical learning
I'm looking for books similar to Introduction to Statistical Learning with Applications in R (ISLR), which is not too rigorous in terms of the mathematical treatment, but still able to provide you the ...
1
vote
0answers
59 views
1
vote
0answers
36 views
how to choose the best fitting function for a data set [closed]
I am trying to find the best fitting function for some data.
I have tried with :
...
1
vote
2answers
110 views
MLE over a large number of parameters
I am working on a model that I am fitting to a medium-sized dataset ($\approx 300,000$ observations). The likelihood I have derived is a mixture model of some Beta distributions which also has the ...
6
votes
0answers
36 views
Simulate correlate random variables with given marginal distribution where one is always larger
Is it possible to simulate pairs of random variables with a given marginal distribution and population correlation where one random variable is larger than the other?
More formally, I need to simulate ...
2
votes
1answer
54 views
(Bayesian) estimation of the underlying population size knowing its upper bound after $x$ draws
Consider you have an initial bag of unique and identifiable items $(1.. K)$. From this bag, someone used an arbitrary criteria to tag $N$ items. You don't know the chosen criteria (which can be ...
1
vote
1answer
369 views
How to calculate z-score from mean and standard deviation in python? [closed]
Is there any way to calculate z-scores from given mean and standard deviation. I know how to do it by hand but couldn't able to find out how to do it in python.
The mean is 81 and standard deviation ...
0
votes
0answers
6 views
How can I transform evidence scores of single-video machine analysis output to confidence scores?
For my research purpose, I need to convert the evidence scores of machine vision output to confidence scores. I found below codes for R in the website:
https://rdrr.io/github/bigpas/face_exp_analysis/...
1
vote
0answers
15 views
Techniques to estimate biasedness and sampling distribution of b0, b1, and $\sigma$ and confidence/prediction intervals when assumptions are violated?
This question is for the linear regression model. Let's say one or more of the assumptions are violated. For example, heteroskedasticity, autocorrelation, or non normality.
I was wondering if there ...
5
votes
1answer
622 views
Why does ordinary least squares have to be linear in the parameters?
I've been looking into linear regression, and on the wikipedia page it says:
"In contrast, non-linear least squares problems generally must be solved by an iterative procedure"
This got me thinking ...
0
votes
1answer
284 views
zero inflated model in R, building the model with pscl, not understanding use of ' | 1'
I have count data with lots of zero. I have done a GLM with poisson distribution, and I think that using the zero inflated model might improve the fit. Now my problem, I have been working with SPSS. ...
