Questions tagged [numerics]
Also known as Numerical Analysis, Numerics aims to provide methods and algorithms for numerical computations.
129
questions
0
votes
0answers
12 views
First difference in logs transformation produces biased results on back-transformation [duplicate]
I have a strongly trended series where the trend appears to be exponential and I believe the errors tend to be proportional to the current value. In order to convert it to a stationary series for ...
0
votes
0answers
17 views
Why Welford’s method is not widely used in statistical library
I just learned about Welford’s method to compute standard deviation using only one pass. However, when I checked the source code of some statistical libraries (e.g., R stats and Numpy), they both use ...
3
votes
1answer
54 views
What is the correct equation for Newton's Method?
Different publications provide different equations for Newton's method or the Newton-Raphson method. In
Giudici, P., Givens, G. H., & Mallick, B. K. (2013). Wiley Series in Computational ...
1
vote
2answers
46 views
I'm so confused about MLE
In maximum likelihood estimation, we maximise the likelihood.
I don't understand how this is possibly: for any reasonable dataset, the likelihood of hitting that EXACT data set is obviously zero! So ...
2
votes
2answers
44 views
Numeric variable with outliers as a categories
I'm working with a dataset that has a few variables that I'm having difficulty trying to preprocess. So one of them is called MENTHLTH where it is a numeric variable.
The point of the variable is to ...
0
votes
0answers
34 views
Seeking algorithms for fast, simple linear regressions
I am working on a project which requires me to work in a proprietary programming language. Unfortunately, this language lacks a matrix algebra library. In this environment, I would like to perform a ...
0
votes
0answers
22 views
Discrete correlation function (sample cross-covariance)
The continuous correlation function for the random variable $A(t)$ at a instant of time $t$ is given by
\begin{equation}
C_{AA}(\tau) =\frac{1}{T} \int_{0}^T d\bar{t} A(\bar{t})A(\bar{t}+\tau)
\end{...
3
votes
3answers
103 views
How to plot $x^{1700}(1-x)^{300}$?
I'm trying to plot a Bernoulli likelihood function on R:
$$x^{1700}(1-x)^{300}$$
But when I try to plot this function on R it looks like this:
I think the maximum should be at 0.85, but it shows me a ...
0
votes
0answers
41 views
Log Sum Exponential Trick On Weibull Mixture [duplicate]
I am trying to evaluate the log-likelihood of a mixture of weibull distributions and am running into problems with the numerical aspect. In short, I have $M$ mixtures and want to evaluate:
$$
\log \...
0
votes
0answers
24 views
how to work with dataset having single target for a multiple rows
I am a beginner in ML/DL. I have majorly worked on a single data file with each row having a target value
Currently, I trying to build a CNN model for a given dataset.
However, I am struggling with ...
6
votes
3answers
691 views
log(1 - softmax(X))? [closed]
Let $\vec X$ be a vector. The $\vec V = \mathrm{logsoftmax}(\vec{X})$ function is defined as:
$$v_i = \ln\left(\frac{e^{x_i}}{\sum_i e^{x_i}}\right)$$
This is provided in machine learning numerical ...
0
votes
0answers
32 views
How to overcome the numerical problem in the Schur complement to update inverse covariance
I'm using the Schur complement (d in my code below) to update a matrix inverse. Suppose that I have square inversible matrix A ...
0
votes
0answers
36 views
Time series reference - Estimating ARIMA models using the Yule-Walker equations and Durbin-Levinson algorithm
I am doing some research related to time-series analysis, and hence was trying to find some good examples of implementations of the common estimation algorithms. For example, I wanted to fit some ...
0
votes
0answers
50 views
Reference request - time series analysis book with numerical algorithms
I am working on some applications of time series, and I wanted to find a book that has the numerical algorithms or pseudocode for computing things like AR models, and ARIMA models, using nonlinear ...
3
votes
0answers
34 views
Why is my QR decomposition updating code numerically off?
I apologize if this is the wrong place for this question; there are a number of potential points of failure each of which suggest either Math StackExchange or StackOverflow or here, but since the ...
2
votes
3answers
177 views
Wrong coefficients in a polynomial fit
I am trying to fit data to a fourth-degree polynomial. I tried this in multiple programs (R, Origin Pro, SigmaPlot), all of which give me a polynomial of the form
$ 40000 -2000x + 40x^2 -0.3x^3 + 0....
2
votes
2answers
153 views
Where is the error in my computation of the wrapped normal distribution density?
Let
$\sigma\in(0,1)$
$$\phi(x):=\frac1{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}}\;\;\;\text{for }x\in\mathbb R$$ and $$\psi(x):=\sum_{k\in\mathbb Z}\phi(x+k)\;\;\;\text{for }x\in\mathbb R$$
$\...
0
votes
0answers
52 views
How should I compute this proposal kernel density?
Let $d\in\mathbb N$ and $$u(x,y):=\beta+(1-\beta)\prod_{i=1}^d\psi(y_i-x_i)\;\;\;\text{for }x,y\in[0,1)^d,$$ where $\beta\in[0,1]$, $$\psi(x):=\sum_{k\in\mathbb Z}\varphi(k+x)\;\;\;\text{for }x\in(-1,...
1
vote
1answer
70 views
Algorithm for simple linear regression that is efficient and numerically stable
I'm developing an application that is fed with continuous data while older data is discarded.
I'm using some algorithms to compute simple linear regression on these data with Perl.
Basically that ...
0
votes
0answers
42 views
Equivalent of log sum exp trick for subtraction [duplicate]
I have two small positive real numbers $u, w$ such that $u > w$. Given $\log(u), \log(w)$ I'd like to find a numerically stable way to calculate $\log(u - w)$.
One possible way of transforming the ...
1
vote
0answers
52 views
Why did the log likelihood decrease with additional parameters?
I'm trying to decide the effect of some factors on the time for an event to happen . Specifically, I am looking at how long it takes for the subject to pass a test (recognize the stimulus) when ...
7
votes
4answers
521 views
Expectation of $\ln(1 + e^x)$, where $x$ is normally distributed
I need to evaluate the following integral:
$$\int_{-\infty}^\infty\mathrm d x \exp\left(-\frac{(x-\mu)^2}{2\nu}\right) \ln(1+e^x)$$
where $\mu$ is a finite real number and $\nu > 0$. This is just ...
1
vote
0answers
10 views
How does the choice of norm affect the condition of a problem?
we know that for a differentiable problem, the absolute condition number is the norm of its jacobian i.e. ||J||. We also know that a well-conditioned problem typically has a small condition number.
...
0
votes
1answer
562 views
Inverse-normal CDF approximation in Excel, Python or R
I read that the implementations of Inverse-normal cumulative distribution function (CDF) /quantile / ppf in R, Python (scipy) and Excel give similar results. However, I can't find the very formulae ...
3
votes
0answers
79 views
Why does lasso return unstable features when using the same data?
I am using scikit-learn to shrink my data set having around 800 features. It is a very noisy data (market and economic data) To my best knowledge, lasso returns same features for the same data set. ...
2
votes
0answers
160 views
Intuition about a coupon problem were we ask for the distribution of the unique coupons when the number of draws is fixed
Alternative viewpoint of the coupon collectors problem
In the coupon collectors problem we draw from a collection of $n$ coupons, with replacement and ask the question how many draws $K$ it takes to ...
3
votes
0answers
196 views
Why is it much quicker to compute ridge regression than regular linear regression?
By my understanding, for a matrix with n samples and p features:
Ridge regression using cholesky takes O(p^3) time
Ordinary linear regression takes O(p^3) time
Singular value decomposition if u, v ...
5
votes
1answer
279 views
How can we numerically compute the autocorrelation of a sample from a Markov chain generated by the Metropolis-Hastings algorithm?
Let $(X_n)_{n\in\mathbb N_0}$ denote a $\mathbb R^d$-valued Markov chain generated by the Metropolis-Hastings algorithm. Suppose I've run the algorithm on a computer and obtained a sample $x_0,\ldots,...
0
votes
0answers
86 views
Model failed to converge in lme4::glmer() when the a factor is centered or releveled
I'm running a mixed-effects model using glmer() function. The modeling works well with R's default dummy coding. But if I center or relevel a factor of 2 levels, the model failed to converge. I am ...
0
votes
0answers
45 views
chi square distribution as a function of other variables
I'm minimizing $\chi^2$ between some data and a function. The expression $\chi^2 = \sum\frac{1}{\sigma_i^2}(y_i-f(A,B))^2$ happens to be analytically solvable, so I have a value for the minimum $\chi^...
3
votes
2answers
609 views
adding a small constant to the diagonals of a matrix to stabilize
I have a large correlation matrix (110x110) with some small eigenvalues (about 20 < 0.1). It has been suggested that adding a constant (about 0.1) to the diagonals will help to stabilize the matrix....
1
vote
1answer
186 views
downsampling a kde / combining kde and histogram
I'm calculating a KDE of one parameter (y, particle density) in bins of another parameter (x, distance from the origin). At ...
1
vote
0answers
31 views
Model or numerical inaccuracy in R package distr
With the goal to have an "outlier"-aware normal distribution, I build a simple univariate mixing model of normal and uniform distributions:
...
1
vote
2answers
188 views
NUTS Drawing samples from slice sampler; how to keep bounds on log scale?
I'm currently working to adapt the No U-Turn Sampler from this paper for a model I'm working on.
The No-U Turn sampler augments the typical hamiltonian system by incorporating a slice variable $u$ ...
1
vote
0answers
214 views
Bayesian Networks - Factor Graphs - Belief Propagation - Numerical stability
I am trying to do inference for a Bayesian Network with discrete probabilities.
I converted the network to a factor graph and implemented the sum-product algorithm (belief propagation). My goal is ...
1
vote
0answers
137 views
Natural Splines and Smoother Matrix
In the context of smoothing splines, one can show that the Reinsch form is given by:
$
\hat{y} = N (N^{T}N +\lambda \Omega)^{-1}N^{T} y
= (I+ \lambda K)^{-1}y
$
where (1) $K = (N^{T})^{-1}\Omega N^{-...
1
vote
0answers
158 views
Moment generating function of a Weibull distribution and root finding heavy and light tailed case
I consider the equation $M_x(v)=1+(1+\beta)\mu$ and I need to find the solution $v>0$ such that the equation is fulfilled.
For this example I consider the moment generating function $M_X(v)$ of a ...
2
votes
3answers
593 views
Estimating correlation matrix using numeric likelihood maximization
I'm performing maximum likelihood estimation on jointly distributed data and I'm having some issues estimating the correlation terms. I am using an approach based on the Cholesky decomposition, but I ...
3
votes
1answer
87 views
Are analytical derivatives unambiguously superior to numerical derivatives in GMM?
I am estimating a non-linear GMM model. In both Stata and R, you need to specify the moment equations and the instruments, but there is no need need to provide analytical derivatives for the estimator ...
1
vote
0answers
106 views
Underflow when estimating marginal likelihood via bridge sampling
I try to use an iterative procedure to estimate the marginal likelihood in a Bayesian setting for model selection. In case you are interested in the specifics of bridge sampling in my application, see ...
3
votes
0answers
335 views
Optimization with/without an analytical gradient
A colleague is optimizing a function (e.g. trying to find the minimum of a function $f(x_1, x_2, \ldots)$). We know the analytical form and it is differentiable. I suggested calculating the ...
1
vote
1answer
981 views
Are mvrnorm() in MASS R package and rmvn() in mgcv R package equivalent?
I am carrying out posterior simulation with GAMs/SCAMs and was wondering if/how the rmvn() function differs in any way from the ...
1
vote
1answer
34 views
numberical implementation of linear regression with “loose variable”
I understand how to solve a linear system $X \beta = y$
the solution is
$\beta = (X^{T}X)^{-1} X^{T} y$
The problem is I could have an entry $\beta_i$ where it has no exposure in $X$. i.e. $X$ has ...
3
votes
0answers
30 views
An urn problem: few red balls, many draws (with replacement)
So, this is a freshman probability problem and I am embarrassed to p[ost it, but I have been up for 35 hours and my brain is broken.
I have an urn with 60,000 white balls and 6 red balls. From this ...
2
votes
0answers
75 views
Dealing with Numerical Issues from Computing Weighted Sample Covariance Matrix
I have vector-valued samples $\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_N$ with normalized weights $w_1, \ldots, w_N$. I'm trying to compute a weighted sample covariance matrix:
$$
\sum_i w_i (\...
0
votes
0answers
1k views
Formula for Inverse T-distribution
I am trying to formulate an expression to calculate the critical value of a T-distribution for a given degrees of freedom. I have done so already for the Normal Distribution by considering the TI-84 ...
5
votes
2answers
514 views
Parameter estimation without an explicit likelihood function
I have a parametric model, some data $y$, and I would like to find a maximum likelihood estimate for the model parameters $\theta$. My usual approach would be to write down the likelihood function $\...
4
votes
0answers
602 views
Choosing the basis functions in a linear regression
I have two random variables $X$ and $Y$ and I'm trying to model $\mathbb{E}[Y|X]$.
To this end, I'd like to pick a collection of functions $f_1, f_2 \dots f_n : \mathbb{R} \to \mathbb{R}$ and then ...
3
votes
2answers
2k views
Deep Learning: Condition Number and Poor Conditioning
I am reading the following section of the book Deep Learning.
Can you provide an intuitive explanation of the above section? I don't quite understand the statement "When this number is large, matrix ...
3
votes
0answers
125 views
Training an Artificial Neural Network with limited-memory Quasi-Newton
I would like to train a simple Artificial Neural Network implementing an algorithm of the class of limited-memory Quasi-Newton.
I read the paper Modified quasi-Newton methods for training neural ...
