# Questions tagged [numerics]

Also known as Numerical Analysis, Numerics aims to provide methods and algorithms for numerical computations.

107 questions
Filter by
Sorted by
Tagged with
31 views

### Why did the log likelihood decrease with additional parameters?

I'm trying to decide the effect of some factors on the time for an event to happen . Specifically, I am looking at how long it takes for the subject to pass a test (recognize the stimulus) when ...
399 views

### Expectation of $\ln(1 + e^x)$, where $x$ is normally distributed

I need to evaluate the following integral: $$\int_{-\infty}^\infty\mathrm d x \exp\left(-\frac{(x-\mu)^2}{2\nu}\right) \ln(1+e^x)$$ where $\mu$ is a finite real number and $\nu > 0$. This is just ...
13 views

### How do you handle large numbers that give infinity in intermediate calculations? [duplicate]

For example, say you need to calculate $\ln \left( \alpha + \beta e^x \right)$ for potentially large $x$. The value itself might be small, but since you need to calculate $e^x$ first, it produces ...
19 views

### Random effects estimates using heuristic / numeric approaches

This is perhaps more a conceptual question. I'm using an heuristic algorithm (ABC: Artificial Bee Colony) to search solutions for a given model that can take additional factors such as numerical and ...
8 views

### How does the choice of norm affect the condition of a problem?

we know that for a differentiable problem, the absolute condition number is the norm of its jacobian i.e. ||J||. We also know that a well-conditioned problem typically has a small condition number. ...
203 views

### Inverse-normal CDF approximation in Excel, Python or R

I read that the implementations of Inverse-normal cumulative distribution function (CDF) /quantile / ppf in R, Python (scipy) and Excel give similar results. However, I can't find the very formulae ...
8 views

### How to measure changes in condition indices over time

I am trying to understand how adding data, one observation at a time, affects the condition indices of a model. A similar question is how adding individual observations affects the principal ...
16 views

### How to quantify rate of convergence in terms of number-of-observations instead of iterations?

I observe discrete points of data, and wish to compute an integral across those points. Since the data is quite sparse, I need to interpolate and extrapolate. There are various approaches in use (...
104 views

### Intuition about a coupon problem were we ask for the distribution of the unique coupons when the number of draws is fixed

Alternative viewpoint of the coupon collectors problem In the coupon collectors problem we draw from a collection of $n$ coupons, with replacement and ask the question how many draws $K$ it takes to ...
79 views

### Why is it much quicker to compute ridge regression than regular linear regression?

By my understanding, for a matrix with n samples and p features: Ridge regression using cholesky takes O(p^3) time Ordinary linear regression takes O(p^3) time Singular value decomposition if u, v ...
143 views

114 views

### Moment generating function of a Weibull distribution and root finding heavy and light tailed case

I consider the equation $M_x(v)=1+(1+\beta)\mu$ and I need to find the solution $v>0$ such that the equation is fulfilled. For this example I consider the moment generating function $M_X(v)$ of a ...
322 views

### Estimating correlation matrix using numeric likelihood maximization

I'm performing maximum likelihood estimation on jointly distributed data and I'm having some issues estimating the correlation terms. I am using an approach based on the Cholesky decomposition, but I ...
58 views

### Are analytical derivatives unambiguously superior to numerical derivatives in GMM?

I am estimating a non-linear GMM model. In both Stata and R, you need to specify the moment equations and the instruments, but there is no need need to provide analytical derivatives for the estimator ...
77 views

### Underflow when estimating marginal likelihood via bridge sampling

I try to use an iterative procedure to estimate the marginal likelihood in a Bayesian setting for model selection. In case you are interested in the specifics of bridge sampling in my application, see ...
197 views

### Optimization with/without an analytical gradient

A colleague is optimizing a function (e.g. trying to find the minimum of a function $f(x_1, x_2, \ldots)$). We know the analytical form and it is differentiable. I suggested calculating the ...
603 views

### Are mvrnorm() in MASS R package and rmvn() in mgcv R package equivalent?

I am carrying out posterior simulation with GAMs/SCAMs and was wondering if/how the rmvn() function differs in any way from the ...
28 views

### numberical implementation of linear regression with “loose variable”

I understand how to solve a linear system $X \beta = y$ the solution is $\beta = (X^{T}X)^{-1} X^{T} y$ The problem is I could have an entry $\beta_i$ where it has no exposure in $X$. i.e. $X$ has ...
27 views

### An urn problem: few red balls, many draws (with replacement)

So, this is a freshman probability problem and I am embarrassed to p[ost it, but I have been up for 35 hours and my brain is broken. I have an urn with 60,000 white balls and 6 red balls. From this ...
57 views

82 views

### Is there a solution for Canonical Correlation Analysis on large sparse matrices?

I'm trying to run CCA over two views which are sparse matrices. The two views are very high dimensional (e.g. 300k, 400k) with 1m samples. CCA needs the input views to be zero mean but I won't be ...
13k views

### Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
52 views

56 views

481 views

### Comparison between analytic gradient and numerical gradient for multivariate normal distribution wrt mean and covariance

The analytic gradients of log multivariate normal distribution wrt mean and covariance matrix can be found at StackExchange post and The gradient of the log-likelihood of normal distributions. I ...
239 views

### Relationship between binomial regression link function and goodness-of-fit tests [now with link to R code]

Some background: A number of papers in the literature (various ones by Hosmer and Lemeshow; Copas; le Cessie and van Houwelingen; Cressie and Read; Osius and Rojek; J. R. Dale) discuss a family of ...
1k views

### Logistic regression and singular Hessian

I've been following along with Andrew Ng's excellent course on Machine Learning (CS 229), and have been working on Problem Set 1. I'm trying to fit a logistic regression model using Newton's Method. ...