Questions tagged [numerics]
Also known as Numerical Analysis, Numerics aims to provide methods and algorithms for numerical computations.
156
questions
0
votes
0
answers
7
views
How to normalize d1 and d2 when using backward finite difference approximation?
My goal is to normalize the first and/or second derivative when approximated from a backward finite difference, where the normalized value is made distinct for some segment of the line (such as ${f'(x)...
- 109
1
vote
1
answer
45
views
MLE for parametric binomial model
I have a model in which $p_i=f(\theta,Z_i)$, where $Z_i$ are iid latent variables distributed with CDF $F_\theta$, and $d_i\sim B(n_i,p_i)$, where $B$ is the binomial distribution. The likelihood ...
- 171
0
votes
0
answers
23
views
How much variance is explained by a given subset of variables (mixed data, both quantitative and qualitative)
My question is very very similiar to the on asked here: How much variance is explained by a given subset of variables?
However, I have mixed data, e.g. both categorical variables (i.e. location) and ...
- 1
1
vote
0
answers
37
views
Comparison of two models with different number of parameters
I want to compare two models, which has different number of parameters. The first model is Arbitrage free Nelson-Siegel model, which has the following equation:
$y_{t}(\tau )=X_{1,t}+X_{2,t}(\frac{1-e^...
- 111
1
vote
0
answers
149
views
Algorithm for Irwin Hall Distribution [closed]
I've been trying to create a function for the Irwin Hall distribution that doesn't face the same issue as the unifed package implementation. Because the function suffers from numerical issues, I ...
- 113
0
votes
0
answers
250
views
Training a physics informed neural network (PINN) in Julia using numerical gradient approximations
i am currently working on a small project which involves solving a pendulum differential equation
$$a \ddot{x} + b \dot{x} + x = 0$$
The idea is to use a physics informed neural network which has two ...
- 321
0
votes
0
answers
67
views
Numerical Stability when Inverse CDF Sampling from Truncated Density
Let $f(x)$ be the pdf of a random variable that we want to truncate to the interval $[a,b]$ and then sample from it. Let $F(x)$ denote the corresponding cdf. We can use inverse cdf sampling and ...
- 681
0
votes
0
answers
22
views
Numerical Solution of two convoluted stable paretian random variables
I am trying to numerically compute the joint density of X and Y, where both are stable paretian distributed random variables with different alphas (1.4 and 1.7).
I can compute the PDF via inversion ...
1
vote
1
answer
57
views
Can (or should) data dominated by two values be treated as categorical?
Problem. I have few data sets containing real values (aka observation and prediction). >85% of data values are oscillating between two values exactly (e.g. 0 or 10), while the rest are real numbers ...
- 31
0
votes
0
answers
61
views
Understanding the Purpose of LogSumExp
According to Wikipedia, the LogSumExp function is a "smooth approximation to the maximum function mainly used by machine learning algorithms." Furthermore,
The LSE function is often ...
- 113
1
vote
1
answer
24
views
A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning
In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said
Again, the implementation of such algorithms can be checked by using
numerical ...
- 303
0
votes
0
answers
32
views
Numerically solving a sparse matrix equation
I want to find the $X$ that solves the matrix equation
$$ AX = B $$
with $A$ and $B$ known - $
A$ and $X$ are rectangular, $A$ is $n \times m $ and $X$ is an $m \times n$, with $m > n$.
(...
2
votes
0
answers
47
views
Recreating equation in R returns different value
I am trying to understand an equation that is used to determine sample size. To do this, I am re-creating the example from a tutorial in R. However, when I re-create the example in R I get a different ...
- 31
2
votes
1
answer
65
views
Establish numerical equivalence of statistical model across software
we are trying to establish numerical equivalence (within reasonable precision) for selected statistical models across programming languages such as SAS, R & Python or even different packages ...
- 21
1
vote
0
answers
390
views
Can I make Stata run lasso faster?
I am trying to run a lasso in Stata.
I have 1.5 million observations and 1700 variables.
Stata is running too slow.
I am in 36th grid after 4 days. And get slower ever grid.
I am using a 98GB Memory, ...
- 51
0
votes
1
answer
61
views
More stable reparametrization of a parameter on $(-1,1)$?
Suppose that a distribution contains a parameter $\theta \in (-1,1)$. I want to reparametrize this model in terms of $\beta = h(\theta) \in (-\infty,\infty)$.
I am considering:
$$h(\theta) = \mbox{...
- 1
4
votes
1
answer
481
views
Is there a closed form approximation for the composition of the Gamma CDF with the inverse Normal CDF?
Given $k$, $\theta$ fixed shape and scale parameters for some Gamma distribution which has a CDF $F$. Let $G^{-1}$ be the inverse CDF of the standard Normal distribution. Consider the composition $H(x)...
- 43
0
votes
0
answers
109
views
Generalized variance of a multivariate normal without calculating determinants
In order to calculate ‘generalized variance’ of a multivariate normal distribution, it is often recommended (e.g., here: https://online.stat.psu.edu/stat505/lesson/1/1.5) to calculate the determinant ...
- 405
0
votes
0
answers
15
views
Can I create a numerical variable from the combination of other numerical variables? [duplicate]
I am working with a dataset that contains limnological parameters. This dataset contains turbidity-related measures in different units and consequently different ranges. Is there a way to create a ...
0
votes
0
answers
42
views
Is there a way to create a numeric variable from a numeric set of measurements with different ranges?
I am working with a dataset that contains limnological parameters. One of them is turbidity-related measures in different units and consequently different ranges. Is there a way to create a numerical ...
0
votes
0
answers
21
views
Are numerical solutions appropriate for inference (eg, estimating variance for confidence intervals)?
For a nicely differentiable objective function, we traditionally always derive the gradients to use for e.g. estimating the variance. (1) Is it common nowadays to use numerical rather than analytical ...
- 5,630
3
votes
1
answer
889
views
Estimate MLE of discrete distribution with two parameters in R [closed]
I want to estimate the MLE of a discrete distribution in R using a numeric method.
My data looks like this:
data1<-c(5,2,2,3,0,2,1 2,4,4,1)
If we assume it ...
- 521
2
votes
1
answer
122
views
Avoid numerical problems with product of probabilities when taking logs or subtracting the maximum is not enough
I have to take draws from the discrete posterior distribution:
$ P(X = x_i |y) \propto P(X = x_i)\prod_{t}^N p(Y_t|X)$
where $P(X = x )$ is the probability mass function of a discrete uniform with ...
- 189
8
votes
3
answers
1k
views
Get accurate eigenvectors, when eigenvalues are minuscule
I have a symmetric matrix A. I'm not able to compute all the eigenvectors accurately, and I believe it is due to the last few eigenvalues for ...
- 117
2
votes
1
answer
86
views
Numerically PCA implements SVD or SVD implements PCA
How do we numerically implement SVD? I confused the numerically implementations between PCA and SVD (who implements who). Since we know that
PCA can be numerically implemented by ...
- 1,245
0
votes
0
answers
127
views
Find the shape parameter for gamma CDF with other parameters known
Is there a reasonably computationally efficient way to find an approximation of the shape parameter (which will always be positive, but not limited to integers) for a gamma CDF (scale = 1) where the ...
- 125
0
votes
1
answer
69
views
Are there examples where errors in numerical analysis has caused wrong decisions in statistics?
I read from statistics books that numerical analysis is not in important role in statistics. Are there any examples in history where numerical approximation has been caused wrong decisions in ...
- 1
1
vote
1
answer
137
views
Is the QR Algorithm guaranteed to compute eigenvectors?
I'm writing some C++ matrix library for hobby.
For computing eigenvalues and eigenvectors, I referred the following "Francis double step QR algorithm":
In particular, page 82 of https://...
- 131
0
votes
0
answers
20
views
First difference in logs transformation produces biased results on back-transformation [duplicate]
I have a strongly trended series where the trend appears to be exponential and I believe the errors tend to be proportional to the current value. In order to convert it to a stationary series for ...
- 3,037
3
votes
1
answer
291
views
What is the correct equation for Newton's Method?
Different publications provide different equations for Newton's method or the Newton-Raphson method. In
Giudici, P., Givens, G. H., & Mallick, B. K. (2013). Wiley Series in Computational ...
- 280
1
vote
2
answers
78
views
I'm so confused about MLE
In maximum likelihood estimation, we maximise the likelihood.
I don't understand how this is possibly: for any reasonable dataset, the likelihood of hitting that EXACT data set is obviously zero! So ...
- 11
2
votes
2
answers
117
views
Numeric variable with outliers as a categories
I'm working with a dataset that has a few variables that I'm having difficulty trying to preprocess. So one of them is called MENTHLTH where it is a numeric variable.
The point of the variable is to ...
- 43
0
votes
0
answers
44
views
Seeking algorithms for fast, simple linear regressions
I am working on a project which requires me to work in a proprietary programming language. Unfortunately, this language lacks a matrix algebra library. In this environment, I would like to perform a ...
- 243
0
votes
0
answers
48
views
Discrete correlation function (sample cross-covariance)
The continuous correlation function for the random variable $A(t)$ at a instant of time $t$ is given by
\begin{equation}
C_{AA}(\tau) =\frac{1}{T} \int_{0}^T d\bar{t} A(\bar{t})A(\bar{t}+\tau)
\end{...
- 101
5
votes
4
answers
187
views
How to plot $x^{1700}(1-x)^{300}$?
I'm trying to plot a Bernoulli likelihood function on R:
$$x^{1700}(1-x)^{300}$$
But when I try to plot this function on R it looks like this:
I think the maximum should be at 0.85, but it shows me a ...
- 53
0
votes
0
answers
180
views
Log Sum Exponential Trick On Weibull Mixture [duplicate]
I am trying to evaluate the log-likelihood of a mixture of weibull distributions and am running into problems with the numerical aspect. In short, I have $M$ mixtures and want to evaluate:
$$
\log \...
- 31
8
votes
3
answers
1k
views
log(1 - softmax(X))? [closed]
Let $\vec X$ be a vector. The $\vec V = \mathrm{logsoftmax}(\vec{X})$ function is defined as:
$$v_i = \ln\left(\frac{e^{x_i}}{\sum_i e^{x_i}}\right)$$
This is provided in machine learning numerical ...
- 3,602
0
votes
0
answers
154
views
Time series reference - Estimating ARIMA models using the Yule-Walker equations and Durbin-Levinson algorithm
I am doing some research related to time-series analysis, and hence was trying to find some good examples of implementations of the common estimation algorithms. For example, I wanted to fit some ...
- 1,502
2
votes
0
answers
128
views
Reference request - time series analysis book with numerical algorithms
I am working on some applications of time series, and I wanted to find a book that has the numerical algorithms or pseudocode for computing things like AR models, and ARIMA models, using nonlinear ...
- 1,502
3
votes
0
answers
76
views
Why is my QR decomposition updating code numerically off?
I apologize if this is the wrong place for this question; there are a number of potential points of failure each of which suggest either Math StackExchange or StackOverflow or here, but since the ...
- 1,313
2
votes
3
answers
834
views
Wrong coefficients in a polynomial fit
I am trying to fit data to a fourth-degree polynomial. I tried this in multiple programs (R, Origin Pro, SigmaPlot), all of which give me a polynomial of the form
$ 40000 -2000x + 40x^2 -0.3x^3 + 0....
user277312
3
votes
2
answers
194
views
Where is the error in my computation of the wrapped normal distribution density?
Let
$\sigma\in(0,1)$
$$\phi(x):=\frac1{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}}\;\;\;\text{for }x\in\mathbb R$$ and $$\psi(x):=\sum_{k\in\mathbb Z}\phi(x+k)\;\;\;\text{for }x\in\mathbb R$$
$\...
- 71
0
votes
0
answers
75
views
How should I compute this proposal kernel density?
Let $d\in\mathbb N$ and $$u(x,y):=\beta+(1-\beta)\prod_{i=1}^d\psi(y_i-x_i)\;\;\;\text{for }x,y\in[0,1)^d,$$ where $\beta\in[0,1]$, $$\psi(x):=\sum_{k\in\mathbb Z}\varphi(k+x)\;\;\;\text{for }x\in(-1,...
- 71
4
votes
0
answers
658
views
Avoid numerical overflow problem in likelihood due to $\exp$
There is a trick called exp-normalization which is used for dealing with overflow for ratios of the type $$\frac{\exp(x_i)}{\sum_j \exp(x_j)} = \frac{\exp(x_i-b)}{\sum_j \exp(x_j-b)}$$ by using the ...
- 6,382
1
vote
1
answer
198
views
Algorithm for simple linear regression that is efficient and numerically stable
I'm developing an application that is fed with continuous data while older data is discarded.
I'm using some algorithms to compute simple linear regression on these data with Perl.
Basically that ...
- 111
0
votes
0
answers
45
views
Equivalent of log sum exp trick for subtraction [duplicate]
I have two small positive real numbers $u, w$ such that $u > w$. Given $\log(u), \log(w)$ I'd like to find a numerically stable way to calculate $\log(u - w)$.
One possible way of transforming the ...
- 43
1
vote
0
answers
147
views
Why did the log likelihood decrease with additional parameters?
I'm trying to decide the effect of some factors on the time for an event to happen . Specifically, I am looking at how long it takes for the subject to pass a test (recognize the stimulus) when ...
- 11
7
votes
4
answers
1k
views
Expectation of $\ln(1 + e^x)$, where $x$ is normally distributed
I need to evaluate the following integral:
$$\int_{-\infty}^\infty\mathrm d x \exp\left(-\frac{(x-\mu)^2}{2\nu}\right) \ln(1+e^x)$$
where $\mu$ is a finite real number and $\nu > 0$. This is just ...
- 3,602
1
vote
0
answers
874
views
System is computationally singular due to small numbers in linearHypothesis
Ok, so here is the code that demonstrate the problem I am referring to:
...
- 193
1
vote
0
answers
12
views
How does the choice of norm affect the condition of a problem?
we know that for a differentiable problem, the absolute condition number is the norm of its jacobian i.e. ||J||. We also know that a well-conditioned problem typically has a small condition number.
...
