Questions tagged [computational-statistics]

Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.

Filter by
Sorted by
Tagged with
47
votes
6answers
30k views

What algorithm is used in linear regression?

I usually hear about "ordinary least squares". Is that the most widely used algorithm used for linear regression? Are there reasons to use a different one?
3
votes
1answer
441 views

Adding very small probabilities - How to compute?

In some problems, probabilities are so small that they are best represented in computational facilities as log-probabilities. Computational problems can arise when you try to add these small ...
25
votes
2answers
13k views

How could stochastic gradient descent save time compared to standard gradient descent?

Standard Gradient Descent would compute gradient for the entire training dataset. ...
52
votes
8answers
3k views

Excel as a statistics workbench

It seems that lots of people (including me) like to do exploratory data analysis in Excel. Some limitations, such as the number of rows allowed in a spreadsheet, are a pain but in most cases don't ...
77
votes
9answers
88k views

What algorithm should I use to detect anomalies on time-series?

Background I'm working in Network Operations Center, we monitor computer systems and their performance. One of the key metrics to monitor is a number of visitors\customers currently connected to our ...
164
votes
21answers
68k views

Does Julia have any hope of sticking in the statistical community?

I recently read a post from R-Bloggers, that linked to this blog post from John Myles White about a new language called Julia. Julia takes advantage of a just-in-time compiler that gives it wicked ...
41
votes
4answers
70k views

How to sample from a normal distribution with known mean and variance using a conventional programming language?

I've never had a course in statistics, so I hope I'm asking in the right place here. Suppose I have only two data describing a normal distribution: the mean $\mu$ and variance $\sigma^2$. I want to ...
6
votes
2answers
907 views

Subtracting very small probabilities - How to compute? [duplicate]

This question is an extension of a related question about adding small probabilities. Suppose you have log-probabilities $\ell_1 \geqslant \ell_2$, where the corresponding probabilities $\exp(\ell_1)$...
4
votes
2answers
7k views

Explanation of the different variable types in statistics?

One thing that has always tripped me up when trying to learn new methods in statistics is understanding what type of features/variables can this method be applied to. The variable types that ...
6
votes
1answer
159 views

Pitfalls in Fitting Nonlinear Models by Transforming to Linearity

Some nonlinear models can be transform to linear models. My understanding is that there might be one-to-one relationship between the estimates of nonlinear model and its linear model form but their ...
2
votes
1answer
142 views

How to derive this version of variance formula

A probably very easy computational question, but I don't really understand how it's done: I try to compute the point estimates of a normal distribution as $N \sim (\beta, \sigma)$. Using the method ...
13
votes
4answers
11k views

Why is gradient descent required?

When we can differentiate the cost function and find parameters by solving equations obtained through partial differentiation with respect to every parameter and find out where the cost function is ...
10
votes
3answers
15k views

Time Series Anomaly Detection with Python

I need to implement anomaly detection on several time-series datasets. I've never done this before and was hoping for some advice. I'm very comfortable with python, so I would prefer the solution be ...
7
votes
1answer
8k views

How do I get the amplitude and phase for sine wave from lm() summary?

A simple sine curve could be written as $\text{amplitude}\cdot\sin(x+\text{phase})$. It can be also written in linear form as $a \cdot \sin(x) + b \cdot \cos(x)$. I run my analysis with R as: ...
12
votes
3answers
1k views

Using computer simulations to better understand statistical concepts at the graduate level

Hi I'm taking a graduate course in Statistics and we've been covering Test statistics, and other concepts. However, I am often able to apply the formulas and develop a sort-of intuition on how stuff ...
10
votes
3answers
17k views

Is it possible in R (or in general) to force regression coefficients to be a certain sign?

I'm working with some real world data and the regression models are yielding some counterintuitive results. Normally I trust the statistics but in reality some of these things can not be true. The ...
7
votes
1answer
4k views

How to split nodes in regression trees

I am looking for a comparison of different regression tree node splitting approaches within the random forest framework. I am looking at the trade-off between ensemble accuracy/reliability (holding ...
6
votes
3answers
3k views

Given two sets, how can I say statistically if they are similar/different

This is a very open ended question. Suppose I have two sets of data samples of the same form, say [item, rating]. Rating is a value on the interval [0,100] and item is a unique identifier given to a ...
5
votes
4answers
4k views

Condition number of covariance matrix

I am interested in generating a covariance matrix of dimension say 100. I managed to get a correlation matrix with finite condition number. To construct a covariance matrix I need to have standard ...
3
votes
1answer
4k views

Using the rejection sampling with the method of inversion

I am hoping to write some rejection algorithm code in R to approximate a $\text{Gamma}(k,\lambda)$ distribution. The problem is more for educational purposes than real-world implementation. Given an ...
3
votes
1answer
158 views

Why apply log to likelihood?

Is there any other reasons beside numerical problems with finite precision system (ieee 754) ? If our computer can have infinite precision, do we still need log likelihood?
0
votes
3answers
3k views

Proof for “The sum of the observed values $Y_i$ equals the sum of the estimated / fitted values $\hat Y_i$” [duplicate]

I needed some help trying to understand why the sum of the observed values $Y_i$ equals the sum of the estimated values $\hat{Y}_i$.
3
votes
2answers
83 views

Is my logistic regression model correct?

I have a factorial design 2*2 (A and B). Both variables with two responses high (coded as 1) and low (coded as 0) and I have a response variable $y$, my logistic model include interaction between A ...
3
votes
4answers
334 views

How to model distributions which are not normally distributed

I would like to model the performance of a rainwater tank, which has a stochastic input (rainfall). The data are the empty volume in the tank at the end of each day. The values are skewed towards the ...
2
votes
1answer
383 views

Area within a given number of standard deviations from given mean

I have a variable with mean value of 18.85 and standard deviation of 1.45. I want to define the area that is covered by 1.45 standard deviations left and 1.45 standard deviations on the right side ...
1
vote
1answer
273 views

What does make SVM a “soft computing” method?

Soft computing is defined in [1] by the capability of "operating with uncertain, imprecise and incomplete information in a manner that reflects human thinking". So, based on my limited understanding, ...
0
votes
0answers
65 views

Getting rid of spikes in sample data [duplicate]

Possible Duplicate: Simple algorithm for online outlier detection of a generic time series How could I get rid of sparky data in a descrete data set, but in a "smoother out" manner? Take for ...
16
votes
9answers
5k views

What books provide an overview of computational statistics as it applies to computer science?

As a software engineer, I'm interested in topics such as statistical algorithms, data mining, machine learning, Bayesian networks, classification algorithms, neural networks, Markov chains, Monte ...
17
votes
2answers
17k views

How to fit a discrete distribution to count data?

I have the following histogram of count data. And I would like to fit a discrete distribution to it. I am not sure how I should go about this. Should I first superimpose a discrete distribution, say ...
16
votes
3answers
4k views

Do some of you use Google Docs spreadsheet to conduct and share your statistical work with others?

I know most of you probably feel that Google Docs is still a primitive tool. It is no Matlab or R and not even Excel. Yet, I am baffled at the power of this web based software that just uses the ...
28
votes
7answers
11k views

Statistics concept to explain why you're less likely to flip the same number of heads as tails, as the number of flips increases?

I'm working on learning probability and statistics by reading a few books and writing some code, and while simulating coin flips I noticed something that struck me as slightly counter to one's naive ...
10
votes
1answer
2k views

Efficient/fast Mahalanobis distance computation

Suppose I have $n$ data points $x_1,\dots,x_n$, each of which is $p$-dimensional. Let $\Sigma$ be the (non-singular) population covariance of these samples. With respect to $\Sigma$, what is the most ...
7
votes
1answer
2k views

What is the difference between 'Laplace approximation' and 'Modified harmonic mean'?

this question is about Bayesian and computational statistics. I am learning them right now, I have two very common output from my software, one is Laplace approximation and the other is Modified ...
9
votes
1answer
3k views

Is functional analysis and hilbert spaces useful in machine learning? If so, how?

I was wondering, how are hilbert spaces and functional analysis useful to machine learning? I thought machine learning was a mix of statistics, computer science and optimization. How does functional ...
9
votes
1answer
10k views

How can I compute a posterior density estimate from a prior and likelihood?

I am trying to understand how to use Bayes' theorem to calculate a posterior but am getting stuck with the computational approach, e.g., in the following case it is not clear to me how to take the ...
13
votes
7answers
1k views

Making sense out of statistics theory and applications

I have recently graduated with my masters degree on medical and biological modeling, accompanied with engineering mathematics as a background. Even though my education program included a significant ...
8
votes
2answers
603 views

Bootstrap vs numerical integration

My understanding of the bootstrap approach is based on Wasserman's framework (almost verbatim): Let $T_n = g(X_1, ..., X_n)$ be a statistic ($X_i$ is the iid sample drawn from distribution $F$). ...
7
votes
1answer
3k views

Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
6
votes
1answer
4k views

What is the current 'standard' for modern statistical computing hardware?

I am in the market for a new system (probably a laptop) that would be be used primarily for Bayesian/MCMC analyses. If I had unlimited funds I would obviously buy very high end hardware and be done ...
10
votes
1answer
426 views

Fast computation/estimation of a low-rank linear system

Linear systems of equations are pervasive in computational statistics. One special system I have encountered (e.g., in factor analysis) is the system $$Ax=b$$ where $$A=D+ B \Omega B^T$$ Here $D$ is ...
8
votes
1answer
5k views

Advice on sensitivity analysis for priors in Bayesian statistics

I'm not clear on how to perform sensitivity analysis on the priors. Many sites have different answers. One site indicates to perform three non-informative, weakly informative and known priors. Another ...
4
votes
2answers
873 views

Extracting power of a power law from data

My question is more about the methodology. Assuming in some experiment we have measured quantity $y$ per each unit of time $x.$ So $y$ and $x$ form our data set here. Moreover, we know that they are ...
4
votes
0answers
935 views

Simulated Annealing Parameter Tuning

My question concerns parameter tuning for simulated annealing (SA). I've the following toy equation $$ y = (x^2+x) \times cos(2x) + 20 \text{ if } x \in (-10, 10) $$ My problem is that the solution ...
3
votes
1answer
2k views

What are the general methods for parameter estimation in statistics?

I have a task to estimate the probability of evolution selection of a given node. The only parameter estimation method I can think of is using the law of large numbers, i.e., use the proportion to ...
2
votes
4answers
170 views

which is the meaning of scatterplot between a pair of 2 consecutive pseudo random numbers with respect to the independence of the sequence?

Pseudo random number generators should give as output random sequences u1, u2, ... that are mutually independent and identically distribuited (iid). Since testing for independence is not easy, the ...
1
vote
1answer
1k views

Acceptance probability for Metropolis-Hastings MCMC on multinomial-Dirichlet model

As an exercise to learn how to manually code MCMC, I've built a Metropolis-Hastings sampler on top of a multinomial-Dirichlet posterior distribution. Since a closed form solution exists, I can compare ...
0
votes
1answer
5k views

RMSE vs. Correlation Coefficient

I am testing my model by 2 different experiments: No test set: I just use cross-validation on the training set. I take a subset of the dataset and use it as a test set (I use the same subset in the ...
7
votes
1answer
3k views

Properties of spectral decomposition

Spectral Decomposition Let $\mathbf{A}$ be a $k\times k$ positive definite matrix with the spectral decomposition $\mathbf{A}=\sum_{i=1}^{k}\lambda_{i}\mathbf{e}_{i}\mathbf{e}_{i}^{\prime}$. Let the ...
5
votes
1answer
63 views

Stable and efficient computation of binomial expectations

Suppose we want to compute the expected value of some function $f(X)$ where $X \sim \text{Bin}(n,\theta)$. Taking $\mathbf{f} = (f_0,...,f_n)$ to be the function values over all possible outcomes of ...
5
votes
1answer
1k views

Cholesky factorization and forward substitution less accurate than inversion?

I recently asked this question asking for an efficient way to compute the Mahalanobis distance (without calculating the inverse). The accepted solution was to use the Cholesky factorization and ...