Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.

learn more… | top users | synonyms

0
votes
0answers
7 views

How to make a loop of transformations on columns (SAS) [on hold]

http://www.math.tau.ac.il/~liadshek/Wide.txt Given this file I want to: a. Run a log transformation on all the columns (for example: GDP1957=log(GDP1957)) b. Run PROC REG (regression) for same year ...
3
votes
0answers
25 views

Defining Groups of 1D Data In a Time Series

I'm trying to analyze the following data. I have a large collection of time collections, and for each time collection, I want to figure out groups of times that are between longer groups. Imagine ...
0
votes
0answers
9 views

Topics needed to learn to for predicting future of a data set?

I am planning to work this summer by writing a code which will eventually become an app, that has the ability to predict what the prices will be atleast 6 month ahead of time. I have a data set of the ...
1
vote
0answers
38 views

Server Requirements for machine learning on 300 million row database [closed]

I am trying to build a logistic regression model using Revolution R 64 bit. My data is 300 million rows by 12 columns. The data is stored on AWS Redshift, but takes about 6 hrs to import and save as ...
0
votes
0answers
8 views

Getting and ordering the data used on a GBM object on R [migrated]

I have a GBM object in R with the underlying data saved on it and I'm having troubles using that data. The problem is that when I run x<-gbmobject$data$x.order ...
0
votes
0answers
6 views

How to find the generalized inverse of a matrix in SageMath?

Given a matrix $A_{m\times n}$, a matrix $G_{n\times m}$ is said to be a generalized inverse of $A$, if it satisfies, $$AGA=A$$ Now, since for a matrix $A$, generalized inverse of $A$ is plenty if ...
0
votes
0answers
16 views

Sampling from Irwin-Hall distribution using triangular distribution

So I need to sample from the Irwin-Hall distribution using rejection sampling with the triangular distribution. I built 2 functions: The first is d_irwin which receives an $x\in supp(g)$ and the n we ...
1
vote
0answers
28 views

Is it ok to ignore data in this instance?

I'm running a program in python I wrote to look at the first spiking time of a neuron in a stochastic network. I'm running about 2500 realizations for each set of parameter values to calculate the ...
1
vote
0answers
34 views

Sampling from Irwin-Hall (Uniform Sum) distribution using rejection sampling

Irwin-Hall Distribution is a probability distribution for a random variable defined as the sum of a number of independent random variables,each having a uniform distribution True or False: The ...
9
votes
2answers
208 views

Inverse covariance matrix vs covariance matrix in PCA

In PCA, does it make a difference if we pick principal components of the inverse covariance matrix OR if we drop eigenvectors of the covariance matrix corresponding to large eigenvalues? This is ...
0
votes
0answers
15 views

Robustness criterion (1978)

I am referring to bradley's journal relating to the robustness criterion. According to his liberal criterion, the simulated type 1 error rate have to between 0.5 alpha < p < 1.5 alpha. since i ...
0
votes
0answers
38 views

Portfolio optimization [migrated]

I am trying to build a portfolio which is optimized with respect to another in R. I am trying to minimize the objective function $$min Var(return_p-return'weight_{bm})$$ with the constraints $$ ...
1
vote
0answers
21 views

Binomial vs inverse binomial transform in obtaining moments

I am trying to obtain raw co-moment matrices from central co-moment matrices and I have found formulas for the transition from central to raw moments here . I have also found the opposite, that is ...
0
votes
0answers
27 views

How to compute the conditional probability distribution with given joint distributions and marginal distribution?

I am trying to compute the conditional distribution p(a|c) with given equation consists of a set of marginal distribution and joint distribution. Variables A,B,C have Markov property A->B->C. $p(a|c) ...
0
votes
1answer
15 views

What tests to analyze the following hypotheses with nominal data?

I have a set of questionnaire data to analyze the working culture for respondents. (There are four existing cultures; a,b,c and d) Variables including Gender, Organization, Public/Private Sector, ...
0
votes
0answers
15 views

Spark big matrix computation

I am trying to see if Spark is fit for my problem. I would like to use kernel methods, as presented in Kernel method. For example, kernel k-means. One important feature of the kernel methods is that ...
1
vote
0answers
34 views

Is this simple paired-sample permutation test valid?

I have about 10 pairs of scores, $(x_1, y_1),~ ...,~(x_{10}, y_{10})$, with all $x_i$ and $y_i$ being between 0.0 and 1.0. I'm interested in testing whether the mean difference over pairs is ...
3
votes
2answers
39 views

Cluster real numbers

I have a set of precise measurements, and what I want to do is count the frequency (how many time it appears) for each value. The problem is that these are very precise measurements and with a naive ...
0
votes
1answer
31 views

value of discrete items when only have their value when sold in groups

I'm doing an analysis for my job and would greatly appreciate any help you can offer! My company sells pallets full of items. Each item has a category (1 category per item, many items per category) ...
0
votes
0answers
24 views

predict mechanical component failure using machine learning

I'm going to do a research project on the possibility to predict mechanical component failure within multidisciplinary system, using sensors data. different sensors are being sampled in different ...
4
votes
1answer
70 views

Posterior distribution of parameter

I am looking at pg. 257 of the textbook "Statistical computing with R" by Maria L. Rizzo, where an example is given of a simple model of stock returns: We have 5 stocks, and at the end of a year of ...
0
votes
0answers
12 views

Best possible statistical approach to compare n pair of variables along a set of genomic intervals

I am dealing a bioinformatics problem and need help in approaching it in the right direction. I have a set of variables that represent different kinds of genomic data and are real valued. I am trying ...
0
votes
0answers
10 views

Find high occurring low values

In numerical data, where outlier is defined not by infrequency but value (this is a real-world data-set where low values would be indicative of noise). Although this is real world data, there is no ...
0
votes
0answers
10 views

creating tidy data sets collaboratively

I am wondering if there is a body of knowledge out there that describes how to best produce tidy data sets in a setting where a lot of people contribute to creating different pieces of the data. So ...
0
votes
0answers
18 views

Testing Random Number Generators(RNG)

There are numerous tests of RNGs. Consider that we tested RNG with lots of different seeds and the output is as follows: with certain seeds the RNG passes the test, with other seeds RNG doesn't pass. ...
0
votes
0answers
14 views

Figuring the algorithm / common calculation on a set

I have a column that is size in MM and another that reflects the change in price for said size. I'm trying to figure out the algorithm/common calculation behind these generated numbers so that I would ...
6
votes
1answer
73 views

Algebraic classifiers, more information?

I have read Algebraic classifiers: a generic approach to fast cross-validation, online training, and parallel training and was amazed by the performance of the derived algorithms. However, it seems ...
0
votes
1answer
30 views

Interpretaion of Logistic Coefficients

All, I ran a logistic Regression on a set of variables both categorical and continuous with a binary event as dependent variable. Now post modelling, I observe a set of categorical variables showing ...
0
votes
0answers
28 views

Calculate the non centrality parameter (ncp) for non central chi squared distribution

I have the 0.975-quantile value (0.191) and the degree of freedom (k=1). How can i calculate the non centrality parameter ? in R if possible. Q0.975(k=1, ncp= ??) = 0.191
1
vote
1answer
54 views

What does make SVM a “soft computing” method?

Soft computing is defined in [1] by the capability of "operating with uncertain, imprecise and incomplete information in a manner that reflects human thinking". So, based on my limited understanding, ...
0
votes
0answers
58 views

Build HMM of text data in R

I'm trying to make my own HMM tagging in R but don't know how to estimate parameter values since the packages I have been working with haven't worked with my data. The latest package I have been ...
2
votes
1answer
55 views

Efficient kernel estimation of density over grid of points

I was wondering whether it is possible to calculate density estimates for $n \times n$ matrix in $o(n^2)$ computational time. Imagine we have a bivariate random vector $(X,Y)$ with a joint density ...
0
votes
0answers
23 views

Finding proper cut off for filtering?

I had a set of data (149876 rows and 1200 columns)and I have found outliers points for each row. Now I want to determine a proper threshold for filtering my outliers. The distribution of my data is ...
1
vote
1answer
34 views

Random unit vectors in $\mathbb{C}^n$ and $\mathbb{R}^{2n}$

Suppose $\mathbf{u}\in\mathbb{C}^n$ is a complex random vector with circular symmetry, uniformly distributed on the unit complex $n$-sphere, so we have $\|\mathbf{u}\|=1$. In other words, $\mathbf{u}$ ...
0
votes
0answers
35 views

help with computing standard score for scarce discretely quantized data

I'm not a stats major and I'd appreciate any help I can get. I've got data with each point defined by a score (between 0-1) and a frequency (0-100%). At 100%, the score is most reliable, at 1% the ...
1
vote
0answers
78 views

calculating PMI for co-occurrences of words

I am in the process of building a question answering system. I am interested in calculating the PMI for words $x$ and $y$ occurring within 5 words of each other in a document. I have the formula and ...
0
votes
1answer
59 views

confusion matrix and Jaccard index computation in O(n) for cluster comparison

My problem: I have two clustering (or partitions) of a set of numbers $\{1,...,n\}$, each with $k$ cluster (or subsets), that i have to compare and, to do that, i'd like to try some different indexes ...
1
vote
0answers
27 views

Multivariate Q-Q plot to compare datasets

I'm trying to compare two datasets, each of two dimensions with unequal size. I tried using Q-Q plot to compare the datasets however I find it's implementation quite difficult. Can anyone suggest some ...
0
votes
2answers
32 views

How to interpret SPSS p-value for one tailed tests

Suppose H0 = Mean 1 - Mean 2 > or = 0 and HA = Mean 1 - Mean 2 < 0. We do an unpaired t-test in SPSS to get our p-value. The SPSS p-value is 0,534. This P-value is two tailed, so we have to devide ...
3
votes
1answer
36 views

Calculate the mean of the minimum of any k samples

Given a fixed list of N numbers from an unknown distribution, and a k (k <= N), say we ...
0
votes
0answers
11 views

Appropriate model to score a test dynamically?

I have a test bank of questions with particular difficulties (say, a number from 1 to 10), and people take "tests" consisting of some subset of those questions. Each question is graded on a 1 to 10 ...
9
votes
2answers
320 views

How to sample from discrete distribution on the non-negative integers?

I have the following discrete distribution, where $\alpha,\beta$ are known constants: $$ p(x;\alpha,\beta) = \frac{\text{Beta}(\alpha+1, \beta+x)}{\text{Beta}(\alpha,\beta)} \;\;\;\;\text{for } x = ...
0
votes
1answer
28 views

Is it possible to apply PCA to the reconstructed data of ICA?

I have read that it is not possible to verify the variance and order of independent component extracted from ICA. So, I reconstructed the spectra from matrix A and ST(independent component) then ...
0
votes
1answer
165 views

How can I calculate the mean and standard deviation given three percentiles using Matlab?

I have the percentiles of 3 values. The percentiles are 5th, 50th and 95th. The values respectively are 340, 365 and 385. How can I find the median and the standard deviation given that data follow a ...
4
votes
1answer
52 views

Duplicating data in ML training

Lets say I have a data set $X_1...X_N$ with the corresponding class of each event $Y(0,1)$. I'm training this set and getting a result for classification. What happens if I simply duplicate the ...
1
vote
0answers
14 views

Prediction of one outcome using another

I have a general question. Suppose I have fit two separate longitudinal mixed effects models (say, model A and model B), they can both be simple and contain only time as a covariate. Say I have ...
1
vote
0answers
12 views

Simulate multinomial logistic regression model [duplicate]

I want to simulate multinomial logistic regression model i have 3 independent continuous variable. I need to simulate a dependent variable which have 5 levels with suitable beta coefficients. Please ...
0
votes
1answer
23 views

Learning a vector function [closed]

The goal is to learn a function $f$ of the form: $f:\mathbb{R} \rightarrow \mathbb{R}^n$, $n \ge 1$ Are there machine learning techniques that can do it? Thank you.
0
votes
0answers
12 views

Gaussian Environment

The perceptron bears a certain relationship to a classical pattern classifier known as the Bayes classifier. When the environment is Gaussian, the Bayes classifier reduces to a linear ...
0
votes
0answers
22 views

Which statistical test for evaluating competency based on perception?

I have this sample of Evaluation Sheet Answers of a company that are going to rank who has the best competency in the work area. The competencies are divided into Managerial Competency, Professional ...