# Questions tagged [computational-statistics]

Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.

79 questions
Filter by
Sorted by
Tagged with
30k views

### What algorithm is used in linear regression?

I usually hear about "ordinary least squares". Is that the most widely used algorithm used for linear regression? Are there reasons to use a different one?
441 views

### Adding very small probabilities - How to compute?

In some problems, probabilities are so small that they are best represented in computational facilities as log-probabilities. Computational problems can arise when you try to add these small ...
13k views

### How could stochastic gradient descent save time compared to standard gradient descent?

Standard Gradient Descent would compute gradient for the entire training dataset. ...
3k views

### Excel as a statistics workbench

It seems that lots of people (including me) like to do exploratory data analysis in Excel. Some limitations, such as the number of rows allowed in a spreadsheet, are a pain but in most cases don't ...
88k views

### What algorithm should I use to detect anomalies on time-series?

Background I'm working in Network Operations Center, we monitor computer systems and their performance. One of the key metrics to monitor is a number of visitors\customers currently connected to our ...
68k views

### Does Julia have any hope of sticking in the statistical community?

I recently read a post from R-Bloggers, that linked to this blog post from John Myles White about a new language called Julia. Julia takes advantage of a just-in-time compiler that gives it wicked ...
70k views

### How to sample from a normal distribution with known mean and variance using a conventional programming language?

I've never had a course in statistics, so I hope I'm asking in the right place here. Suppose I have only two data describing a normal distribution: the mean $\mu$ and variance $\sigma^2$. I want to ...
907 views

### Subtracting very small probabilities - How to compute? [duplicate]

This question is an extension of a related question about adding small probabilities. Suppose you have log-probabilities $\ell_1 \geqslant \ell_2$, where the corresponding probabilities $\exp(\ell_1)$...
7k views

### Explanation of the different variable types in statistics?

One thing that has always tripped me up when trying to learn new methods in statistics is understanding what type of features/variables can this method be applied to. The variable types that ...
159 views

### Pitfalls in Fitting Nonlinear Models by Transforming to Linearity

Some nonlinear models can be transform to linear models. My understanding is that there might be one-to-one relationship between the estimates of nonlinear model and its linear model form but their ...
142 views

### How to derive this version of variance formula

A probably very easy computational question, but I don't really understand how it's done: I try to compute the point estimates of a normal distribution as $N \sim (\beta, \sigma)$. Using the method ...
11k views

### Why is gradient descent required?

When we can differentiate the cost function and find parameters by solving equations obtained through partial differentiation with respect to every parameter and find out where the cost function is ...
15k views

### Time Series Anomaly Detection with Python

I need to implement anomaly detection on several time-series datasets. I've never done this before and was hoping for some advice. I'm very comfortable with python, so I would prefer the solution be ...
8k views

### How do I get the amplitude and phase for sine wave from lm() summary?

A simple sine curve could be written as $\text{amplitude}\cdot\sin(x+\text{phase})$. It can be also written in linear form as $a \cdot \sin(x) + b \cdot \cos(x)$. I run my analysis with R as: ...
1k views

### Using computer simulations to better understand statistical concepts at the graduate level

Hi I'm taking a graduate course in Statistics and we've been covering Test statistics, and other concepts. However, I am often able to apply the formulas and develop a sort-of intuition on how stuff ...
17k views

### Is it possible in R (or in general) to force regression coefficients to be a certain sign?

I'm working with some real world data and the regression models are yielding some counterintuitive results. Normally I trust the statistics but in reality some of these things can not be true. The ...
4k views

### How to split nodes in regression trees

I am looking for a comparison of different regression tree node splitting approaches within the random forest framework. I am looking at the trade-off between ensemble accuracy/reliability (holding ...
3k views

### Given two sets, how can I say statistically if they are similar/different

This is a very open ended question. Suppose I have two sets of data samples of the same form, say [item, rating]. Rating is a value on the interval [0,100] and item is a unique identifier given to a ...
4k views

### Condition number of covariance matrix

I am interested in generating a covariance matrix of dimension say 100. I managed to get a correlation matrix with finite condition number. To construct a covariance matrix I need to have standard ...
4k views

### Using the rejection sampling with the method of inversion

I am hoping to write some rejection algorithm code in R to approximate a $\text{Gamma}(k,\lambda)$ distribution. The problem is more for educational purposes than real-world implementation. Given an ...
158 views

### Why apply log to likelihood?

Is there any other reasons beside numerical problems with finite precision system (ieee 754) ? If our computer can have infinite precision, do we still need log likelihood?
3k views

### Proof for “The sum of the observed values $Y_i$ equals the sum of the estimated / fitted values $\hat Y_i$” [duplicate]

I needed some help trying to understand why the sum of the observed values $Y_i$ equals the sum of the estimated values $\hat{Y}_i$.
83 views

### Is my logistic regression model correct?

I have a factorial design 2*2 (A and B). Both variables with two responses high (coded as 1) and low (coded as 0) and I have a response variable $y$, my logistic model include interaction between A ...
334 views

### How to model distributions which are not normally distributed

I would like to model the performance of a rainwater tank, which has a stochastic input (rainfall). The data are the empty volume in the tank at the end of each day. The values are skewed towards the ...
383 views

### Area within a given number of standard deviations from given mean

I have a variable with mean value of 18.85 and standard deviation of 1.45. I want to define the area that is covered by 1.45 standard deviations left and 1.45 standard deviations on the right side ...
273 views

### What does make SVM a “soft computing” method?

Soft computing is defined in [1] by the capability of "operating with uncertain, imprecise and incomplete information in a manner that reflects human thinking". So, based on my limited understanding, ...
65 views

### Getting rid of spikes in sample data [duplicate]

Possible Duplicate: Simple algorithm for online outlier detection of a generic time series How could I get rid of sparky data in a descrete data set, but in a "smoother out" manner? Take for ...
5k views

### What books provide an overview of computational statistics as it applies to computer science?

As a software engineer, I'm interested in topics such as statistical algorithms, data mining, machine learning, Bayesian networks, classification algorithms, neural networks, Markov chains, Monte ...
17k views

### How to fit a discrete distribution to count data?

I have the following histogram of count data. And I would like to fit a discrete distribution to it. I am not sure how I should go about this. Should I first superimpose a discrete distribution, say ...
4k views

### Do some of you use Google Docs spreadsheet to conduct and share your statistical work with others?

I know most of you probably feel that Google Docs is still a primitive tool. It is no Matlab or R and not even Excel. Yet, I am baffled at the power of this web based software that just uses the ...
11k views

### Statistics concept to explain why you're less likely to flip the same number of heads as tails, as the number of flips increases?

I'm working on learning probability and statistics by reading a few books and writing some code, and while simulating coin flips I noticed something that struck me as slightly counter to one's naive ...
2k views

### Efficient/fast Mahalanobis distance computation

Suppose I have $n$ data points $x_1,\dots,x_n$, each of which is $p$-dimensional. Let $\Sigma$ be the (non-singular) population covariance of these samples. With respect to $\Sigma$, what is the most ...
2k views

### What is the difference between 'Laplace approximation' and 'Modified harmonic mean'?

this question is about Bayesian and computational statistics. I am learning them right now, I have two very common output from my software, one is Laplace approximation and the other is Modified ...
3k views

### Is functional analysis and hilbert spaces useful in machine learning? If so, how?

I was wondering, how are hilbert spaces and functional analysis useful to machine learning? I thought machine learning was a mix of statistics, computer science and optimization. How does functional ...
10k views

### How can I compute a posterior density estimate from a prior and likelihood?

I am trying to understand how to use Bayes' theorem to calculate a posterior but am getting stuck with the computational approach, e.g., in the following case it is not clear to me how to take the ...
1k views

### Making sense out of statistics theory and applications

I have recently graduated with my masters degree on medical and biological modeling, accompanied with engineering mathematics as a background. Even though my education program included a significant ...
603 views

### Bootstrap vs numerical integration

My understanding of the bootstrap approach is based on Wasserman's framework (almost verbatim): Let $T_n = g(X_1, ..., X_n)$ be a statistic ($X_i$ is the iid sample drawn from distribution $F$). ...
3k views

### Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
4k views

### What is the current 'standard' for modern statistical computing hardware?

I am in the market for a new system (probably a laptop) that would be be used primarily for Bayesian/MCMC analyses. If I had unlimited funds I would obviously buy very high end hardware and be done ...
426 views

### Fast computation/estimation of a low-rank linear system

Linear systems of equations are pervasive in computational statistics. One special system I have encountered (e.g., in factor analysis) is the system $$Ax=b$$ where $$A=D+ B \Omega B^T$$ Here $D$ is ...
5k views

### Advice on sensitivity analysis for priors in Bayesian statistics

I'm not clear on how to perform sensitivity analysis on the priors. Many sites have different answers. One site indicates to perform three non-informative, weakly informative and known priors. Another ...
873 views

### Extracting power of a power law from data

My question is more about the methodology. Assuming in some experiment we have measured quantity $y$ per each unit of time $x.$ So $y$ and $x$ form our data set here. Moreover, we know that they are ...
935 views

### Simulated Annealing Parameter Tuning

My question concerns parameter tuning for simulated annealing (SA). I've the following toy equation $$y = (x^2+x) \times cos(2x) + 20 \text{ if } x \in (-10, 10)$$ My problem is that the solution ...
2k views

### What are the general methods for parameter estimation in statistics?

I have a task to estimate the probability of evolution selection of a given node. The only parameter estimation method I can think of is using the law of large numbers, i.e., use the proportion to ...
170 views

### which is the meaning of scatterplot between a pair of 2 consecutive pseudo random numbers with respect to the independence of the sequence?

Pseudo random number generators should give as output random sequences u1, u2, ... that are mutually independent and identically distribuited (iid). Since testing for independence is not easy, the ...
1k views

### Acceptance probability for Metropolis-Hastings MCMC on multinomial-Dirichlet model

As an exercise to learn how to manually code MCMC, I've built a Metropolis-Hastings sampler on top of a multinomial-Dirichlet posterior distribution. Since a closed form solution exists, I can compare ...
5k views

### RMSE vs. Correlation Coefficient

I am testing my model by 2 different experiments: No test set: I just use cross-validation on the training set. I take a subset of the dataset and use it as a test set (I use the same subset in the ...
3k views

### Properties of spectral decomposition

Spectral Decomposition Let $\mathbf{A}$ be a $k\times k$ positive definite matrix with the spectral decomposition $\mathbf{A}=\sum_{i=1}^{k}\lambda_{i}\mathbf{e}_{i}\mathbf{e}_{i}^{\prime}$. Let the ...
Suppose we want to compute the expected value of some function $f(X)$ where $X \sim \text{Bin}(n,\theta)$. Taking $\mathbf{f} = (f_0,...,f_n)$ to be the function values over all possible outcomes of ...