# Tagged Questions

Refers to the interface of statistics and computing; the use of algorithms and software for statistical purposes.

51 views

### What are common ill-conditioned problems/mistakes in linear regression? [on hold]

Many problems are ill-conditioned and cannot be reliably solved using double precision computer systems. Here is what I know that could happen in linear model, that the problem is ill-conditioned and ...
109 views

### How do iterative methods for solving maximizing likelihood problems work?

Does anyone know about the computational iteration processes for maximum likelihood estimation? If these set of equations cannot be solved practically then how does the computer solve them?
19 views

### Numerical optimization of Monte-Carlo simulated moments

I hope not to post an off-topic related question here as it is related to programming, however, I am faced to a specific problem which I 1) tackle in a statistically oriented language and 2) which I ...
34 views

### computation of square root of matrix

I am studying the Canonical correlation analysis(CCA) the formula https://en.wikipedia.org/wiki/Canonical_correlation is involving the -1/2 degree of a matrix. My question is: how can i do the ...
29 views

### Since we use Bootsrap to approximate the SE, can we use Bootstrap to find prediction errors?

Instead of using Cross-validation, or K-fold Cross-validation, can we use Bootstrap to generate random samples and use one of them as test set, and others as training set?
39 views

### how to find similarity based on only one column value

I am not sure if it is possible and that is why I am asking the question here. I have a data looks like below ...
36 views

### Random Forest explanation

I am having trouble understanding Random Forest, especially some terms. What is a node what is node size? What are ...
14 views

### Unbalanced classification problem: Can more data be more harmful?

In a two-class problem, I have 1000 raw data to train my model, which 900 belongs to class 1 and 100 belongs to class 2. If I add another 500 data into the pool but they all belong to class 1, will ...
23 views

### Negative estimated values [closed]

I'm using Weka API to make a prediction with SVR. My dataset contains 499 records. In the outpout of the program, I get some negative predicted values. Is that due to problem of Overfitting? If yes, ...
147 views

### What is the best way to detect repetition in xyz data for purposes of splitting data?

I'll use this picture to explain What I want to do is define some patterns as trained patterns. Then given data I want to be able to determine if the pattern exists in the dataset, and if it does ...
302 views

### The value of an Effect Size

I calculated a Cohen's d value of d= -2.1. I understand that there are small, medium, and large effect sizes. But in my case the d value is negative? Would it still be considered large since abs(-2.1)...
29 views

### What is your configuration to do statistical analysis on large data sets?

The stat packages which I've been using are not very well suited for working with large data sets, i.e. more than 1 billion records. They all tend to try loading all into memory, and install one one ...
28 views

### Is it possible to add up the accuracy rates of 2 predictors?

Weather channel 1 has a 65% accuracy rate of predicting tomorrows weather Weather channel 2 has a 59% accuracy rate of predicting tomorrows weather Is it somehow possible to take into account ...
12 views

### How to measure the impact of a product on the global revenue?

We have products daily revenue and when each product started being sold by the company. We also have different break-down of product revenue by location, store, sales person, etc. We need to measure ...
55 views

### Does it make sense to cluster based on Euclidean distances between rows of a cosine matrix?

I calculated the Cosine distances for binary data and got the relations between different variables. I need to cluster them. I tried passing the cosine matrix directly to the (clustering) function ...
33 views

### How to compute correlation for two data sets?

I am a CS student and naive in statistics. I have 2 related data sets, to which I want to find correlation. Let me explain the case first: This is a question related to computer architecture. One ...
60 views

16 views

### What tests to analyze the following hypotheses with nominal data?

I have a set of questionnaire data to analyze the working culture for respondents. (There are four existing cultures; a,b,c and d) Variables including Gender, Organization, Public/Private Sector, ...
19 views

### Spark big matrix computation

I am trying to see if Spark is fit for my problem. I would like to use kernel methods, as presented in Kernel method. For example, kernel k-means. One important feature of the kernel methods is that ...
43 views

### Is this simple paired-sample permutation test valid?

I have about 10 pairs of scores, $(x_1, y_1),~ ...,~(x_{10}, y_{10})$, with all $x_i$ and $y_i$ being between 0.0 and 1.0. I'm interested in testing whether the mean difference over pairs is ...
48 views

### Cluster real numbers

I have a set of precise measurements, and what I want to do is count the frequency (how many time it appears) for each value. The problem is that these are very precise measurements and with a naive ...
33 views

### value of discrete items when only have their value when sold in groups

I'm doing an analysis for my job and would greatly appreciate any help you can offer! My company sells pallets full of items. Each item has a category (1 category per item, many items per category) ...
55 views

### predict mechanical component failure using machine learning

I'm going to do a research project on the possibility to predict mechanical component failure within multidisciplinary system, using sensors data. different sensors are being sampled in different ...
72 views

### Posterior distribution of parameter

I am looking at pg. 257 of the textbook "Statistical computing with R" by Maria L. Rizzo, where an example is given of a simple model of stock returns: We have 5 stocks, and at the end of a year of ...
16 views

### Best possible statistical approach to compare n pair of variables along a set of genomic intervals

I am dealing a bioinformatics problem and need help in approaching it in the right direction. I have a set of variables that represent different kinds of genomic data and are real valued. I am trying ...
11 views

### Find high occurring low values

In numerical data, where outlier is defined not by infrequency but value (this is a real-world data-set where low values would be indicative of noise). Although this is real world data, there is no ...
10 views

### creating tidy data sets collaboratively

I am wondering if there is a body of knowledge out there that describes how to best produce tidy data sets in a setting where a lot of people contribute to creating different pieces of the data. So ...
18 views

### Testing Random Number Generators(RNG)

There are numerous tests of RNGs. Consider that we tested RNG with lots of different seeds and the output is as follows: with certain seeds the RNG passes the test, with other seeds RNG doesn't pass. ...
14 views

### Figuring the algorithm / common calculation on a set

I have a column that is size in MM and another that reflects the change in price for said size. I'm trying to figure out the algorithm/common calculation behind these generated numbers so that I would ...
77 views

I have read Algebraic classifiers: a generic approach to fast cross-validation, online training, and parallel training and was amazed by the performance of the derived algorithms. However, it seems ...
31 views

### Interpretaion of Logistic Coefficients

All, I ran a logistic Regression on a set of variables both categorical and continuous with a binary event as dependent variable. Now post modelling, I observe a set of categorical variables showing ...