Skip to main content

Questions tagged [numerics]

Also known as Numerical Analysis, Numerics aims to provide methods and algorithms for numerical computations.

Filter by
Sorted by
Tagged with
0 votes
0 answers
32 views

Estimating correlation parameter from known value of bivariate normal distribution

I want to estimate the correlation parameter $\rho$ using the following expression taken from this paper (equation 10 on page 17): $$ \hat{s}^2+\hat{\mu}^2=N_2(N^{-1}(\hat{\mu}),N^{-1}(\hat{\mu}), \...
MysteriousBrit's user avatar
2 votes
1 answer
80 views

How to do log subtract (just like logsumexp) with probabilities? [closed]

To subtract a small probability from another, this answer has constraint on log probabilities l1 > l2: Subtracting very small probabilities - How to compute? but I need a function that works for ...
monotonic's user avatar
  • 201
2 votes
1 answer
55 views

What is a numerically stable way to generate an exponential distribution that properly yields very large, low-probability values in Excel and C++?

I have sets of sampled data with the following statistics: Because the mean is so close to the min, and because of our understanding of the process that generated the samples, we are treating the ...
All The Rage's user avatar
3 votes
1 answer
4k views

Understanding the advantages of BF16 vs. FP16 in mixed precision training

Brain float (BF16) and 16-bit floating point (FP16) both require 2 bytes of memory, but in contrast to FP16, BF16 allows to represent a much larger numerical range than FP16, so under-/overflows won't ...
Green绿色's user avatar
0 votes
0 answers
37 views

Numerical quadrature for Pareto distribution

I would like to numerically evaluate an integral of the following type, when evaluating $f(x)$ at any given point is numerically costly: $$ \int_{x_m}^\infty x^{-\alpha}f(x) \, dx, \quad \alpha >1, ...
spellard's user avatar
0 votes
0 answers
30 views

Gradient descent residual

I've implemented the gradient descent method for finding roots of a system of nonlinear equations and I am wondering how the residual is determined? Is the residual simply the Euclidean norm (2-norm) ...
blov's user avatar
  • 1
5 votes
2 answers
158 views

Approximating the standard normal density with the logistic density: How to numerically optimize $\infty$-norm?

Let's say that we want to use the logistic distribution as an approximation to the standard normal density. As the location parameter of the logistic distribution is $0$, the scale parameter $s$ is ...
COOLSerdash's user avatar
0 votes
1 answer
46 views

Get samples from a known log density

I have two distributions $p_a$ and $p_b$ and I want to sample from $p_c$, defined via the log density $$ \log p_c(x) = (1+w) \log p_a(x) - w \log p_b(x) $$ or via the desnity $$ p_c(x) = \frac{ p_a(x)^...
fabian789's user avatar
  • 111
2 votes
1 answer
49 views

MLE for parametric binomial model

I have a model in which $p_i=f(\theta,Z_i)$, where $Z_i$ are iid latent variables distributed with CDF $F_\theta$, and $d_i\sim B(n_i,p_i)$, where $B$ is the binomial distribution. The likelihood ...
user2520938's user avatar
1 vote
0 answers
47 views

Comparison of two models with different number of parameters

I want to compare two models, which has different number of parameters. The first model is Arbitrage free Nelson-Siegel model, which has the following equation: $y_{t}(\tau )=X_{1,t}+X_{2,t}(\frac{1-e^...
Shelley's user avatar
  • 111
1 vote
0 answers
282 views

Algorithm for Irwin Hall Distribution [closed]

I've been trying to create a function for the Irwin Hall distribution that doesn't face the same issue as the unifed package implementation. Because the function suffers from numerical issues, I ...
user1329307's user avatar
0 votes
0 answers
97 views

Numerical Stability when Inverse CDF Sampling from Truncated Density

Let $f(x)$ be the pdf of a random variable that we want to truncate to the interval $[a,b]$ and then sample from it. Let $F(x)$ denote the corresponding cdf. We can use inverse cdf sampling and ...
yrx1702's user avatar
  • 710
1 vote
1 answer
65 views

Can (or should) data dominated by two values be treated as categorical?

Problem. I have few data sets containing real values (aka observation and prediction). >85% of data values are oscillating between two values exactly (e.g. 0 or 10), while the rest are real numbers ...
Art's user avatar
  • 31
1 vote
1 answer
27 views

A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning

In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said Again, the implementation of such algorithms can be checked by using numerical ...
zzzhhh's user avatar
  • 333
0 votes
0 answers
32 views

Numerically solving a sparse matrix equation

I want to find the $X$ that solves the matrix equation $$ AX = B $$ with $A$ and $B$ known - $
A$ and $X$ are rectangular, $A$ is $n \times m $ and $X$ is an $m \times n$, with $m > n$. (...
Christopher Turnbull's user avatar
2 votes
0 answers
47 views

Recreating equation in R returns different value

I am trying to understand an equation that is used to determine sample size. To do this, I am re-creating the example from a tutorial in R. However, when I re-create the example in R I get a different ...
user3200293's user avatar
3 votes
1 answer
68 views

Establish numerical equivalence of statistical model across software

we are trying to establish numerical equivalence (within reasonable precision) for selected statistical models across programming languages such as SAS, R & Python or even different packages ...
Vineet's user avatar
  • 31
1 vote
0 answers
497 views

Can I make Stata run lasso faster? [closed]

I am trying to run a lasso in Stata. I have 1.5 million observations and 1700 variables. Stata is running too slow. I am in 36th grid after 4 days. And get slower ever grid. I am using a 98GB Memory, ...
Valt Yo's user avatar
  • 51
0 votes
1 answer
72 views

More stable reparametrization of a parameter on $(-1,1)$?

Suppose that a distribution contains a parameter $\theta \in (-1,1)$. I want to reparametrize this model in terms of $\beta = h(\theta) \in (-\infty,\infty)$. I am considering: $$h(\theta) = \mbox{...
Calip's user avatar
  • 1
4 votes
1 answer
584 views

Is there a closed form approximation for the composition of the Gamma CDF with the inverse Normal CDF?

Given $k$, $\theta$ fixed shape and scale parameters for some Gamma distribution which has a CDF $F$. Let $G^{-1}$ be the inverse CDF of the standard Normal distribution. Consider the composition $H(x)...
Apen13's user avatar
  • 43
0 votes
0 answers
180 views

Generalized variance of a multivariate normal without calculating determinants

In order to calculate ‘generalized variance’ of a multivariate normal distribution, it is often recommended (e.g., here: https://online.stat.psu.edu/stat505/lesson/1/1.5) to calculate the determinant ...
macleginn's user avatar
  • 415
0 votes
0 answers
15 views

Can I create a numerical variable from the combination of other numerical variables? [duplicate]

I am working with a dataset that contains limnological parameters. This dataset contains turbidity-related measures in different units and consequently different ranges. Is there a way to create a ...
Patricia Nunes's user avatar
0 votes
0 answers
44 views

Is there a way to create a numeric variable from a numeric set of measurements with different ranges?

I am working with a dataset that contains limnological parameters. One of them is turbidity-related measures in different units and consequently different ranges. Is there a way to create a numerical ...
Patricia Nunes's user avatar
0 votes
0 answers
21 views

Are numerical solutions appropriate for inference (eg, estimating variance for confidence intervals)?

For a nicely differentiable objective function, we traditionally always derive the gradients to use for e.g. estimating the variance. (1) Is it common nowadays to use numerical rather than analytical ...
Sam Weisenthal's user avatar
3 votes
1 answer
1k views

Estimate MLE of discrete distribution with two parameters in R [closed]

I want to estimate the MLE of a discrete distribution in R using a numeric method. My data looks like this: data1<-c(5,2,2,3,0,2,1 2,4,4,1) If we assume it ...
user10386405's user avatar
3 votes
1 answer
129 views

Avoid numerical problems with product of probabilities when taking logs or subtracting the maximum is not enough

I have to take draws from the discrete posterior distribution: $ P(X = x_i |y) \propto P(X = x_i)\prod_{t}^N p(Y_t|X)$ where $P(X = x )$ is the probability mass function of a discrete uniform with ...
Giorgetto's user avatar
  • 319
8 votes
3 answers
1k views

Get accurate eigenvectors, when eigenvalues are minuscule

I have a symmetric matrix A. I'm not able to compute all the eigenvectors accurately, and I believe it is due to the last few eigenvalues for ...
Mich55's user avatar
  • 117
2 votes
1 answer
95 views

Numerically PCA implements SVD or SVD implements PCA

How do we numerically implement SVD? I confused the numerically implementations between PCA and SVD (who implements who). Since we know that PCA can be numerically implemented by ...
user6703592's user avatar
  • 1,345
0 votes
0 answers
150 views

Find the shape parameter for gamma CDF with other parameters known

Is there a reasonably computationally efficient way to find an approximation of the shape parameter (which will always be positive, but not limited to integers) for a gamma CDF (scale = 1) where the ...
Brent's user avatar
  • 125
0 votes
1 answer
80 views

Are there examples where errors in numerical analysis has caused wrong decisions in statistics?

I read from statistics books that numerical analysis is not in important role in statistics. Are there any examples in history where numerical approximation has been caused wrong decisions in ...
curious's user avatar
1 vote
1 answer
172 views

Is the QR Algorithm guaranteed to compute eigenvectors?

I'm writing some C++ matrix library for hobby. For computing eigenvalues and eigenvectors, I referred the following "Francis double step QR algorithm": In particular, page 82 of https://...
frozenca's user avatar
  • 131
0 votes
0 answers
20 views

First difference in logs transformation produces biased results on back-transformation [duplicate]

I have a strongly trended series where the trend appears to be exponential and I believe the errors tend to be proportional to the current value. In order to convert it to a stationary series for ...
andrewH's user avatar
  • 3,157
3 votes
1 answer
320 views

What is the correct equation for Newton's Method?

Different publications provide different equations for Newton's method or the Newton-Raphson method. In Giudici, P., Givens, G. H., & Mallick, B. K. (2013). Wiley Series in Computational ...
Tea Tree's user avatar
  • 280
1 vote
2 answers
94 views

I'm so confused about MLE

In maximum likelihood estimation, we maximise the likelihood. I don't understand how this is possibly: for any reasonable dataset, the likelihood of hitting that EXACT data set is obviously zero! So ...
Name's user avatar
  • 11
2 votes
2 answers
129 views

Numeric variable with outliers as a categories

I'm working with a dataset that has a few variables that I'm having difficulty trying to preprocess. So one of them is called MENTHLTH where it is a numeric variable. The point of the variable is to ...
CD'A's user avatar
  • 43
0 votes
0 answers
44 views

Seeking algorithms for fast, simple linear regressions

I am working on a project which requires me to work in a proprietary programming language. Unfortunately, this language lacks a matrix algebra library. In this environment, I would like to perform a ...
John L.'s user avatar
  • 243
0 votes
0 answers
53 views

Discrete correlation function (sample cross-covariance)

The continuous correlation function for the random variable $A(t)$ at a instant of time $t$ is given by \begin{equation} C_{AA}(\tau) =\frac{1}{T} \int_{0}^T d\bar{t} A(\bar{t})A(\bar{t}+\tau) \end{...
sined's user avatar
  • 101
5 votes
4 answers
189 views

How to plot $x^{1700}(1-x)^{300}$?

I'm trying to plot a Bernoulli likelihood function on R: $$x^{1700}(1-x)^{300}$$ But when I try to plot this function on R it looks like this: I think the maximum should be at 0.85, but it shows me a ...
Eric's user avatar
  • 53
0 votes
0 answers
209 views

Log Sum Exponential Trick On Weibull Mixture [duplicate]

I am trying to evaluate the log-likelihood of a mixture of weibull distributions and am running into problems with the numerical aspect. In short, I have $M$ mixtures and want to evaluate: $$ \log \...
Kieran108's user avatar
8 votes
3 answers
1k views

log(1 - softmax(X))? [closed]

Let $\vec X$ be a vector. The $\vec V = \mathrm{logsoftmax}(\vec{X})$ function is defined as: $$v_i = \ln\left(\frac{e^{x_i}}{\sum_i e^{x_i}}\right)$$ This is provided in machine learning numerical ...
a06e's user avatar
  • 4,450
0 votes
0 answers
202 views

Time series reference - Estimating ARIMA models using the Yule-Walker equations and Durbin-Levinson algorithm

I am doing some research related to time-series analysis, and hence was trying to find some good examples of implementations of the common estimation algorithms. For example, I wanted to fit some ...
krishnab's user avatar
  • 1,522
2 votes
0 answers
134 views

Reference request - time series analysis book with numerical algorithms

I am working on some applications of time series, and I wanted to find a book that has the numerical algorithms or pseudocode for computing things like AR models, and ARIMA models, using nonlinear ...
krishnab's user avatar
  • 1,522
3 votes
0 answers
77 views

Why is my QR decomposition updating code numerically off?

I apologize if this is the wrong place for this question; there are a number of potential points of failure each of which suggest either Math StackExchange or StackOverflow or here, but since the ...
cgmil's user avatar
  • 1,373
2 votes
3 answers
998 views

Wrong coefficients in a polynomial fit

I am trying to fit data to a fourth-degree polynomial. I tried this in multiple programs (R, Origin Pro, SigmaPlot), all of which give me a polynomial of the form $ 40000 -2000x + 40x^2 -0.3x^3 + 0....
user avatar
4 votes
2 answers
201 views

Where is the error in my computation of the wrapped normal distribution density?

Let $\sigma\in(0,1)$ $$\phi(x):=\frac1{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}}\;\;\;\text{for }x\in\mathbb R$$ and $$\psi(x):=\sum_{k\in\mathbb Z}\phi(x+k)\;\;\;\text{for }x\in\mathbb R$$ $\...
0xbadf00d's user avatar
  • 303
0 votes
0 answers
76 views

How should I compute this proposal kernel density?

Let $d\in\mathbb N$ and $$u(x,y):=\beta+(1-\beta)\prod_{i=1}^d\psi(y_i-x_i)\;\;\;\text{for }x,y\in[0,1)^d,$$ where $\beta\in[0,1]$, $$\psi(x):=\sum_{k\in\mathbb Z}\varphi(k+x)\;\;\;\text{for }x\in(-1,...
0xbadf00d's user avatar
  • 303
4 votes
0 answers
775 views

Avoid numerical overflow problem in likelihood due to $\exp$

There is a trick called exp-normalization which is used for dealing with overflow for ratios of the type $$\frac{\exp(x_i)}{\sum_j \exp(x_j)} = \frac{\exp(x_i-b)}{\sum_j \exp(x_j-b)}$$ by using the ...
tomka's user avatar
  • 6,634
1 vote
1 answer
271 views

Algorithm for simple linear regression that is efficient and numerically stable

I'm developing an application that is fed with continuous data while older data is discarded. I'm using some algorithms to compute simple linear regression on these data with Perl. Basically that ...
U. Windl's user avatar
  • 111
0 votes
0 answers
47 views

Equivalent of log sum exp trick for subtraction [duplicate]

I have two small positive real numbers $u, w$ such that $u > w$. Given $\log(u), \log(w)$ I'd like to find a numerically stable way to calculate $\log(u - w)$. One possible way of transforming the ...
marcusy's user avatar
  • 43
1 vote
0 answers
162 views

Why did the log likelihood decrease with additional parameters?

I'm trying to decide the effect of some factors on the time for an event to happen . Specifically, I am looking at how long it takes for the subject to pass a test (recognize the stimulus) when ...
Hongyu Li's user avatar