Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Also known as Numerical Analysis, Numerics aims to provide methods and algorithms for numerical computations.

0
votes
1answer
21 views

Given unnormalized weights in logs, how do I compute normalized weights in logs?

Setting Given a set of positive weights $\{w_i\}_{i=1}^n$, I can normalize them by computing $$W_i = \frac{w_i}{\sum_{j=1}^nw_j}\quad \forall i=1,...,n.$$ Easy enough. But for numerical reasons, it ...
0
votes
0answers
14 views

Example of method of moments solved numerically?

I have come across many examples where the method of moments estimators can be found in closed form (Exponential, Normal, and etcetera). However, there are cases where there is no closed form moments, ...
0
votes
0answers
7 views

how to systematically choose the thresholding in least square solver, for solving ill-posed least square problem?

I have a system $Ax =b$, where $A\in\mathbb{R}^{300\times 200}$, but $rank(A)=70$. And I know the true solution. I tried standard least square solver such as ...
0
votes
0answers
13 views

Easy way to sample a Bayes posterior distribution of stable distributions?

I have a markov chain $P(x_{i+1}|x_i)=\rho(x_{i+1} ; \alpha,\beta,c, x_i)$, where $\rho$ is the stable distribution with mean $x_i$. I'm interested in fixing $x_1$ and $x_3$, and sampling an $x_2$ ...
1
vote
0answers
37 views

Natural Splines and Smoother Matrix

In the context of smoothing splines, one can show that the Reinsch form is given by: $ \hat{y} = N (N^{T}N +\lambda \Omega)^{-1}N^{T} y = (I+ \lambda K)^{-1}y $ where (1) $K = (N^{T})^{-1}\Omega N^{-...
1
vote
0answers
30 views

Moment generating function of a Weibull distribution and root finding heavy and light tailed case

I consider the equation $M_x(v)=1+(1+\beta)\mu$ and I need to find the solution $v>0$ such that the equation is fulfilled. For this example I consider the moment generating function $M_X(v)$ of a ...
0
votes
0answers
49 views

Estimating correlation matrix using numeric likelihood maximization

I'm performing maximum likelihood estimation on jointly distributed data and I'm having some issues estimating the correlation terms. I am using an approach based on the Cholesky decomposition, but I ...
2
votes
1answer
23 views

Are analytical derivatives unambiguously superior to numerical derivatives in GMM?

I am estimating a non-linear GMM model. In both Stata and R, you need to specify the moment equations and the instruments, but there is no need need to provide analytical derivatives for the estimator ...
1
vote
0answers
26 views

Underflow when estimating marginal likelihood via bridge sampling

I try to use an iterative procedure to estimate the marginal likelihood in a Bayesian setting for model selection. In case you are interested in the specifics of bridge sampling in my application, see ...
0
votes
0answers
38 views

Numerical Solution for Logistic Regression

Reading the Numba tutorial, I found a neat numerical solution for logistic regression ...
2
votes
1answer
108 views

Are mvrnorm() in MASS R package and rmvn() in mgcv R package equivalent?

I am carrying out posterior simulation with GAMs/SCAMs and was wondering if/how the rmvn() function differs in any way from the ...
0
votes
0answers
16 views

Discretize numeric attribute values

I have a numeric attribute (e.g. revenue attribute), I must discretize its values before I use WEKA tool, but based on what I must divide them into categories? Since this may affect the classification ...
1
vote
1answer
26 views

numberical implementation of linear regression with “loose variable”

I understand how to solve a linear system $X \beta = y$ the solution is $\beta = (X^{T}X)^{-1} X^{T} y$ The problem is I could have an entry $\beta_i$ where it has no exposure in $X$. i.e. $X$ has ...
3
votes
0answers
26 views

An urn problem: few red balls, many draws (with replacement)

So, this is a freshman probability problem and I am embarrassed to p[ost it, but I have been up for 35 hours and my brain is broken. I have an urn with 60,000 white balls and 6 red balls. From this ...
2
votes
0answers
43 views

Dealing with Numerical Issues from Computing Weighted Sample Covariance Matrix

I have vector-valued samples $\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_N$ with normalized weights $w_1, \ldots, w_N$. I'm trying to compute a weighted sample covariance matrix: $$ \sum_i w_i (\...
1
vote
0answers
36 views

How many possibilities to divide 12 elements into 4 buckets of 3 elements [closed]

Say we have possibilities: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12 4 buckets that can contain 3 elements How may possibilities are there to divide the 12 elements over the 4 buckets (where each ...
0
votes
0answers
280 views

Formula for Inverse T-distribution

I am trying to formulate an expression to calculate the critical value of a T-distribution for a given degrees of freedom. I have done so already for the Normal Distribution by considering the TI-84 ...
5
votes
2answers
118 views

Parameter estimation without an explicit likelihood function

I have a parametric model, some data $y$, and I would like to find a maximum likelihood estimate for the model parameters $\theta$. My usual approach would be to write down the likelihood function $\...
4
votes
0answers
128 views

Choosing the basis functions in a linear regression

I have two random variables $X$ and $Y$ and I'm trying to model $\mathbb{E}[Y|X]$. To this end, I'd like to pick a collection of functions $f_1, f_2 \dots f_n : \mathbb{R} \to \mathbb{R}$ and then ...
1
vote
0answers
365 views

Deep Learning: Condition Number and Poor Conditioning

I am reading the following section of the book Deep Learning. Can you provide an intuitive explanation of the above section? I don't quite understand the statement "When this number is large, matrix ...
2
votes
0answers
42 views

Training an Artificial Neural Network with limited-memory Quasi-Newton

I would like to train a simple Artificial Neural Network implementing an algorithm of the class of limited-memory Quasi-Newton. I read the paper Modified quasi-Newton methods for training neural ...
0
votes
0answers
34 views

Numerical Error in Multivariate Gaussian Likelihood Calculation

I've ran into an interesting issue in MATLAB computing multivariate Gaussian likelihoods. While I believe this method to be correct mathematically (and computationally efficient) it has produced ...
0
votes
0answers
25 views

How much statisticians use numerical analysis while comparing critical values?

I found that many statistical software uses numerical methods to compute P-values and critical values of a test. How can I be sure that if I do some statistical analysis and found a P value that is ...
4
votes
2answers
109 views

Why is two-sided gradient checking more accurate? [closed]

In week 5 of Andrew Ng's Machine Learning course, he gives the formulae for gradient checking: One-sided difference: $\dfrac{\partial}{\partial\Theta}J(\Theta) \approx \dfrac{J(\Theta + \epsilon) - ...
0
votes
0answers
52 views

Integral of probit likelihood

I'm currently trying to implement the Heckman method for estimating the dynamic probit panel model (Original paper can be found here). I'm trying to implement it according a paper of Stewart (2006), ...
1
vote
0answers
212 views

GMM Estimation and convergence problem

I try to minimize an unweighted moment function $G(\theta)$ given by $G(\theta) = \bar{g}(\theta)'\bar{g}(\theta) $. $g(\theta,x_i)$ contains the specified moment conditions, where we state $E(g(\...
2
votes
1answer
205 views

Moments of the truncated normal distribution (univariate) away from the mean

I need to compute the mean and variance of the truncated normal distribution. For simplicity, let us focus on a standard normal, since the general case can be reduced to this. The PDF is given by: $$...
1
vote
0answers
53 views

Is there a solution for Canonical Correlation Analysis on large sparse matrices?

I'm trying to run CCA over two views which are sparse matrices. The two views are very high dimensional (e.g. 300k, 400k) with 1m samples. CCA needs the input views to be zero mean but I won't be ...
24
votes
4answers
7k views

Why does Andrew Ng prefer to use SVD and not EIG of covariance matrix to do PCA?

I am studying PCA from Andrew Ng's Coursera course and other materials. In the Stanford NLP course cs224n's first assignment, and in the lecture video from Andrew Ng, they do singular value ...
0
votes
1answer
48 views

Distribution of infinite sum $\sum_{t=0}^{\infty} \epsilon_t r^t $

In my current statistics course we're being taught about time series, and in this context we came across sums like this: $$\sum_{t=1}^{\infty} \epsilon_t r^t \quad \epsilon_t\sim \text{WN}(0,\sigma^2) ...
0
votes
1answer
73 views

Method to compute empirical derivative about some point

I have black-box access to some function and I want to compute the derivative about the point X. Is there a method that does this?
4
votes
1answer
2k views

Softmax overflow [closed]

Waiting the next course of Andrew Ng on Coursera, I'm trying to program on Python a classifier with the softmax function on the last layer to have the different probabilities. However, when I try to ...
4
votes
1answer
475 views

What is the best way to apply the log-sum-exp trick in this situation?

I am aware of the "log-sum-exp" trick for calculating the logarithm of sums that handles overflow and underflow issues. However, I would like to know more about how it works. In particular, I am ...
1
vote
1answer
41 views

Numerical method to find x [closed]

I am carrying out a simulation study using the cumulative distribution function $$ F(x)=\frac{4x\arctan\frac{x}{a}}{a^2(π-2)}$$ Now I need to get x subject in order to carry out the simulation study. ...
1
vote
0answers
44 views

Proving an equivalence relation by using numerical methods such as Gaussian quadrature

The background is residual useful life prediction. The following is my problem description. Degradation signal path: $r(t)=\phi+\theta t$ , where $\phi$ is assumed to be the same for all units,and $\...
4
votes
0answers
138 views

What is the fastest way to compute PC1 scores, without performing the whole PCA?

I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points ...
2
votes
0answers
330 views

Comparison between analytic gradient and numerical gradient for multivariate normal distribution wrt mean and covariance

The analytic gradients of log multivariate normal distribution wrt mean and covariance matrix can be found at StackExchange post and The gradient of the log-likelihood of normal distributions. I ...
6
votes
1answer
198 views

Relationship between binomial regression link function and goodness-of-fit tests [now with link to R code]

Some background: A number of papers in the literature (various ones by Hosmer and Lemeshow; Copas; le Cessie and van Houwelingen; Cressie and Read; Osius and Rojek; J. R. Dale) discuss a family of ...
2
votes
0answers
629 views

Logistic regression and singular Hessian

I've been following along with Andrew Ng's excellent course on Machine Learning (CS 229), and have been working on Problem Set 1. I'm trying to fit a logistic regression model using Newton's Method. ...
0
votes
0answers
51 views

Accurate tail probabilities (p-values) in MASS package

An R package was reporting a p-value for my dataset to be zero. Digging into it, I found that the p-values were computed in MASS:::summary.loglm : ...
7
votes
2answers
883 views

Is it possible to have pearson correlation coefficient values < -1 or values > 1?

I am trying to calculate the Pearson correlation coefficient according this formula over a large dataset: Mostly, my values are between -1 and 1, but sometimes I get weird numbers like: ...
3
votes
1answer
869 views

Why do prcomp() and eigen(cov()) in R return different signs of PCA eigenvectors?

I understand the sign of the eigen vectors / PCA rotations can be positive or negative (see here or here). But I am curious why the following two approaches yield different results, from numerical ...
1
vote
0answers
23 views

What analysis can be performed for categorical and numeric factors and their interactions on numeric response variable?

What analysis can be performed to see the effects of categorical and numeric factors and their interactions on a continuous or numeric response variable? It seems that I need to use ANCOVA, but not ...
-2
votes
2answers
195 views

plotting a factorial function in R

I have problem trying to plot the following function in R: The R code I'm using is: ...
1
vote
1answer
177 views

Interpolate a CDF to get an interpolated hazard rate, or interpolate the hazard rate directly?

My problem is that I need to do an interpolation. Eventually, I will work on the hazard rate, but I do not know if it is better to interpolate the CDF or the hazard rate. Let me explain better. I've ...
2
votes
1answer
254 views

Monotonicity of special case of Kullback-Leibler divergence

I have two discrete distributions $\tau$ and $\rho$ with the same support $\Omega$. I'm considering a weighted mixture of these distributions described by the following function: $$ f(w) = (1-w) \cdot ...
5
votes
2answers
3k views

Kullback-Leibler Divergence

I tried to implement a numerical estimate of the Kullback-Leibler Divergence for two samples. To debug the implementation draw the samples from two normal distributions $\mathcal N (0,1)$ and $\...
4
votes
2answers
137 views

Does rounding introduce variance into estimates?

It is often recommended to round parameter estimates to avoid suggesting more precision than the data really have, e.g. here. I understand rounding does not introduce bias, as long as an unbiased ...
0
votes
0answers
35 views

Good algorithm to compute the sample mean and variance [duplicate]

I am looking for good (fast, memory efficient, ideally one-pass) algorithms to compute the sample mean and variance. I have found a technical report by Chan, Golub, and LeVeque from 1983 with ...
1
vote
1answer
506 views

Numerically stable correlation coefficient calculation

I have been trying to calculate the correlation coefficient $(\rho)$ of two variables, and noticed that in cases where either $var(X)$ or $var(Y)$ are very small, the correlation coefficient ...