Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.

learn more… | top users | synonyms

0
votes
0answers
17 views

Calculate the non centrality parameter (ncp) for non central chi squared distribution

I have the 0.975-quantile value (0.191) and the degree of freedom (k=1). How can i calculate the non centrality parameter ? in R if possible. Q0.975(k=1, ncp= ??) = 0.191
0
votes
0answers
25 views

What does make SVM a “soft computing” method?

Soft computing is defined in [1] by the capability of "operating with uncertain, imprecise and incomplete information in a manner that reflects human thinking". So, based on my limited understanding, ...
0
votes
0answers
33 views

Build HMM of text data in R

I'm trying to make my own HMM tagging in R but don't know how to estimate parameter values since the packages I have been working with haven't worked with my data. The latest package I have been ...
2
votes
1answer
18 views

Efficient kernel estimation of density over grid of points

I was wondering whether it is possible to calculate density estimates for $n \times n$ matrix in $o(n^2)$ computational time. Imagine we have a bivariate random vector $(X,Y)$ with a joint density ...
0
votes
0answers
13 views

Finding proper cut off for filtering?

I had a set of data (149876 rows and 1200 columns)and I have found outliers points for each row. Now I want to determine a proper threshold for filtering my outliers. The distribution of my data is ...
1
vote
1answer
33 views

Random unit vectors in $\mathbb{C}^n$ and $\mathbb{R}^{2n}$

Suppose $\mathbf{u}\in\mathbb{C}^n$ is a complex random vector with circular symmetry, uniformly distributed on the unit complex $n$-sphere, so we have $\|\mathbf{u}\|=1$. In other words, $\mathbf{u}$ ...
0
votes
0answers
25 views

help with computing standard score for scarce discretely quantized data

I'm not a stats major and I'd appreciate any help I can get. I've got data with each point defined by a score (between 0-1) and a frequency (0-100%). At 100%, the score is most reliable, at 1% the ...
1
vote
0answers
26 views

calculating PMI for co-occurrences of words

I am in the process of building a question answering system. I am interested in calculating the PMI for words $x$ and $y$ occurring within 5 words of each other in a document. I have the formula and ...
0
votes
1answer
28 views

confusion matrix and Jaccard index computation in O(n) for cluster comparison

My problem: I have two clustering (or partitions) of a set of numbers $\{1,...,n\}$, each with $k$ cluster (or subsets), that i have to compare and, to do that, i'd like to try some different indexes ...
1
vote
0answers
25 views

Multivariate Q-Q plot to compare datasets

I'm trying to compare two datasets, each of two dimensions with unequal size. I tried using Q-Q plot to compare the datasets however I find it's implementation quite difficult. Can anyone suggest some ...
0
votes
2answers
25 views

How to interpret SPSS p-value for one tailed tests

Suppose H0 = Mean 1 - Mean 2 > or = 0 and HA = Mean 1 - Mean 2 < 0. We do an unpaired t-test in SPSS to get our p-value. The SPSS p-value is 0,534. This P-value is two tailed, so we have to devide ...
3
votes
1answer
33 views

Calculate the mean of the minimum of any k samples

Given a fixed list of N numbers from an unknown distribution, and a k (k <= N), say we ...
0
votes
0answers
11 views

Appropriate model to score a test dynamically?

I have a test bank of questions with particular difficulties (say, a number from 1 to 10), and people take "tests" consisting of some subset of those questions. Each question is graded on a 1 to 10 ...
9
votes
2answers
313 views

How to sample from discrete distribution on the non-negative integers?

I have the following discrete distribution, where $\alpha,\beta$ are known constants: $$ p(x;\alpha,\beta) = \frac{\text{Beta}(\alpha+1, \beta+x)}{\text{Beta}(\alpha,\beta)} \;\;\;\;\text{for } x = ...
0
votes
1answer
25 views

Is it possible to apply PCA to the reconstructed data of ICA?

I have read that it is not possible to verify the variance and order of independent component extracted from ICA. So, I reconstructed the spectra from matrix A and ST(independent component) then ...
0
votes
1answer
41 views

How can I calculate the mean and standard deviation given three percentiles using Matlab?

I have the percentiles of 3 values. The percentiles are 5th, 50th and 95th. The values respectively are 340, 365 and 385. How can I find the median and the standard deviation given that data follow a ...
4
votes
1answer
46 views

Duplicating data in ML training

Lets say I have a data set $X_1...X_N$ with the corresponding class of each event $Y(0,1)$. I'm training this set and getting a result for classification. What happens if I simply duplicate the ...
1
vote
0answers
12 views

Prediction of one outcome using another

I have a general question. Suppose I have fit two separate longitudinal mixed effects models (say, model A and model B), they can both be simple and contain only time as a covariate. Say I have ...
1
vote
0answers
12 views

Simulate multinomial logistic regression model [duplicate]

I want to simulate multinomial logistic regression model i have 3 independent continuous variable. I need to simulate a dependent variable which have 5 levels with suitable beta coefficients. Please ...
0
votes
1answer
23 views

Learning a vector function [closed]

The goal is to learn a function $f$ of the form: $f:\mathbb{R} \rightarrow \mathbb{R}^n$, $n \ge 1$ Are there machine learning techniques that can do it? Thank you.
0
votes
0answers
10 views

Gaussian Environment

The perceptron bears a certain relationship to a classical pattern classifier known as the Bayes classifier. When the environment is Gaussian, the Bayes classifier reduces to a linear ...
0
votes
0answers
18 views

Which statistical test for evaluating competency based on perception?

I have this sample of Evaluation Sheet Answers of a company that are going to rank who has the best competency in the work area. The competencies are divided into Managerial Competency, Professional ...
2
votes
1answer
63 views

How to specify when a level shift begins and ends or in the case of data series with multiple level shifts how to id when one level shift beings/ends?

I am working on forecasting airport delays the data looks like this It looks like there is a structural break around 2004 where theres a huge increase and then a huge decrease around 2009. I am ...
0
votes
1answer
56 views

Probability with unknown sample mean and unknown standard deviation

I am studying statistics in a textbook. I have successfully answers two questions about normal distribution. Here are the questions Credit scores are three-digit numbers used by lenders when ...
1
vote
1answer
31 views

Probability statistics [closed]

I pick sports for fun with my friends. we pick 5 sports picks everyday. what is the probability of going at least 3/5 for 50 days in a row? How would you set this up
0
votes
0answers
53 views

Calculating the Chi Squared from combinations of subgroups

Let's say I can calculate the chi squared p-value of different categorical variables and summarise them into a table like so: Initial Data: ...
27
votes
7answers
4k views

Statistics concept to explain why you're less likely to flip the same number of heads as tails, as the number of flips increases?

I'm working on learning probability and statistics by reading a few books and writing some code, and while simulating coin flips I noticed something that struck me as slightly counter to one's naive ...
0
votes
1answer
17 views

Scalability comparison with the help of regression

I created an algorithm and I tested it against a current algorithm. The results are in this form: ...
0
votes
0answers
26 views

Implementing Canonical Correlation Analysis in practice

I'm trying to implement Canonical Correlation Analysis using the Eigen template header c++ library to better my understanding of both the library and the particular statistical technique. So far, the ...
3
votes
1answer
25 views

Reducing variance of Monte Carlo estimates via a known relationship

Here's a simplification of the actual problem, but I believe it captures the essence of it. Say we are interested in estimating the expectation of a stochastic time series. We can simulate the time ...
3
votes
0answers
21 views

differences between fake computer-generated data & real data

generally, is there a/some formula(s) or algorithm(s) to prove that a data set is a fake one generated using computer and not by field work? more specific, recently I got a research questionnaire data ...
-1
votes
1answer
60 views

How do I compute a realization of h(x) given its PDF and covariance?

I've added a picture of what I want to compute. In the nomenclature of the picture, I want to compute a realization of y(x) given the known distributions and constants. Let's say y(x), random ...
0
votes
1answer
52 views

correct feature selection for stable feature subset

I was just reading "Resampling Strategies for Model Assessment and Selection" by Richard Simon (Springer), and he says the following on page 183: "Data analysts are sometimes tempted to use the ...
2
votes
1answer
50 views

Computation of average number of hops for an atom to reach a certain distance

I apologize if this has been asked before. I tried searching all previous posts to look at different forms of this problem (such as random 2d walks on a lattice, relation to Isling's work, etc.). I ...
0
votes
1answer
35 views

Empirical distribution using R [closed]

I have a data frame that shows the actions and status of a project in two fields.I am going to use Empirical distribution to find the real probability not theoretical probability. I found that I can ...
0
votes
1answer
36 views

R coding and the gamma distribution

I have never used R before this assignment so it's completely new to me as is coding in general. My assignment has 3 questions. The first question I tried to construct the distribution the only way I ...
3
votes
2answers
106 views

Probability Question with random sample

i am trying to solve some probability question. Here are they poll found that 73% of households own digital cameras. A random sample of 9 households is selected A. What is the probability that 2 of ...
3
votes
1answer
50 views

Books for statistical computing course?

I'm taking an introductory graduate course in statistical programming. I usually like to read textbooks along with my coursework, but the professor doesn't have any suggestions for books matching the ...
0
votes
0answers
17 views

AVERAGES OF SETS

Am not really sure whether its handled in this field;but my question is: Suppose there are different sets of random numbers provided. Each set has exactly 16 digits ranging from 0 to 9,all randomly ...
3
votes
2answers
83 views

Simulation for the random vector (X,Y) with density f(x,y,a)

I need to generate data from a random vector with joint density $(X,Y)$ with density: $$ \frac{x^\alpha(x+y+2)}{x+y+1}e^{-x-y}~~~~~,\alpha,x,y> 0 $$ Do you have any hints on how to start?
2
votes
2answers
226 views

What is the interpretation of positive log-likelihood for discrete time series data?

I use auto.arima function to model the below provided time series data. At the end of the analysis, the best model is given as ARIMA(1,2,1). The log- likelihood=93.69 is positive which is unusual. It ...
2
votes
2answers
76 views

How to simulate random observations from a specific distribution?

I am asking for a general approach about how to construct algorithms akin to, for example, the rnorm function in R given that one has, say, a closed-form ...
1
vote
0answers
32 views

Measure of similarity between two multivariate samples

I have two samples from a multivariate distribution. One is approximate and the other is exact. My aim is to measure the similarity between the approximate sample and the exact sample. In particular, ...
4
votes
0answers
52 views

Kurtosis, bias, unbiased and statistics

I apologize ahead of time if this is too vague or meta to be a valid question. I've been looking at Algorithms (Sedgewick & Wayne). They define a class stdstats. In that they define min, max, ...
0
votes
1answer
105 views
+50

efficient “dot” product of two sparse vectors with uncertainty

I'm interested in extensions or analogs of the vector dot product that apply to sparse vectors in the case of uncertainty in the abscissa. The vectors I deal with are often of large (100,000 or ...
0
votes
0answers
30 views

Question regarding bias-reduced logistic regression and its outcome

I have a data set shown below, the 1st, 2nd,3rd column are dependent variable(dv), and 2 independent variables (iv1 & iv2) respectively, I expected the regression coefficient of the "iv1" shows a ...
3
votes
1answer
75 views

Why do floating point random number methods exclude 1.0 in distribution

I've been 'playing about' with pseudo random generators in C#, while studying Algorithms (Sedgewick & Wayne). I've noticed that the BCL Random class generates distributions in a 'half-open' way ...
1
vote
0answers
35 views

Statistical test with small samples

I want to compare 3 groups of samples. Each sample contains 5 measurements. I want to know if the samples differ significantly. First I wanted to do a ANOVA, but my samples are not normal distributed. ...
1
vote
0answers
30 views

Augmented frequency scale of Term frequency as it appear on Wikipedia

The third definition of the tf(t, d) in Wikipedia is: I was wondering, why would we want to scale to the range of 0.5..1?
0
votes
0answers
28 views

Objective evaluation of subjective data points and normalization

I want to write an access database that takes in inputs from instructors ranking a student on their performance in a handful of categories. (1 - below average, 2 - average, 3 - above average). They ...