# Questions tagged [partitioning]

A partition is an assignment of every element of a set into 1 & only 1 subset w/ no empty subsets. A common instance of partitioning in statistics is the partitioning of sums of squares for F-tests.

114 questions
Filter by
Sorted by
Tagged with
17 views

### Clustering a distance matrix with k-medoids

For a symmetric distance matrix that I want to cluster, I performed several cluster algorithms: MDS into k-Means DBSCAN OPTICS k-Medoids (the one I'm having trouble with) Now, I would like to know ...
15 views

### Use inference on the whole data as a prior to regularize estimates in a subgroup

Suppose that the probability of a certain condition in the data $D$ is $\theta$ which can be estimated by updating a Beta prior as $Pr(\theta|s, N) = Beta(\alpha + s, \beta + N - s)$ with $s$ being ...
11 views

### Bayesian evaluation if partitioning is justified for a dataset

I'd like to comparare whether partitioning of a dataset is justified. The data is categorical with two levels and the fitted parameter is the prevalence of positives for a certain condition in each ...
9 views

### How can I apply partitioning chi square test?

I have a table that has 4 groups and 5 comparisons. Groups are: monthly, several time annually, rarely and never. Comparisons are: stage 0, 1, 2, 3, 4. Chi square test were performed. There is a ...
8 views

### Representing partitioned N-dimensional input data

I am working on N-dimensional inputs and single dimensional response data sets. I want to get local picture of slopes in my response and hence, I am dividing my input space into local regions by ...
22 views

### Learn how to bisect 2d data to arrive at 1d relationships

I want to find a way to fit a series of curves throught 2d data. I have a hypothsis of how this can be done (I will explain it below). I would appreciate if anyone could suggest whether this technique ...
5 views

### When/where do I correlation analysis during cross validation using training/test (holdout) partitions?

I'm doing Multiple Regression Right now I'm using a training partition to do [partial] correlation analysis. Then once done I split the data up into folds and do backwards step cross validation and ...
34 views

### If likelihood is statistically sufficient, how could some cases have no sufficient statistics?

Almost all elementary texts clarify that in some cases minimal sufficient statistics might not exist. However, it seems that the likelihood itself induces a partition that essentially provides a ...
16 views

### A line through the origin that bounds a proportion of points in a scatterplot

Suppose I have a scatterplot in a box with 0 < x < x1 and y1 < y < y2. Let 0 < prp < 1. Is there an efficient way to find a line that passes through the origin, and that has prp ...
926 views

### Data partitioning for spatial data

I am constructing different configurations of a Random Forest in order to investigate the influence of well-design variables and location, on the first-year production volumes of shale oil wells, ...
45 views

### Exact distribution of number of transitions in a binary vector of length $n$

I am trying to derive an analytic solution for the exact distribution of the number of transitions in a binary vector of length $n$ conditional on the observed number of $1$'s and $0$'s and under the ...
72 views

### variation partitioning with a GAMM model including an auto-correlation structure in R

I would like to undertake variation partitioning in a GAM framework in R, as described here: https://r.789695.n4.nabble.com/variance-explained-by-each-term-in-a-GAM-td836513.html However, my gam ...
49 views

### Finding the probability of winning on or before n trials? [closed]

I have two teams A and B, the probability that A wins each hole is p, the probability that B wins each hole is q and the probability that neither wins is r. What is the probability of A winning on or ...
37 views

### In this concrete example of applying sklearn knn (with kd_tree) on Iris Data Set, how many partitions are there?

The k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input consists of the k closest training examples in the ...
27 views

### Recursive partitioning tree vs neural network model

I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression ...
104 views

### Does the multinomial coefficient create combinations or permutations?

I am looking at this formula I just want to make sure I get it. For example, if I want to split a fifty-two card deck into all the possible combinations of four, thirteen card hands, then I get: 52!...
157 views

### default plot for mob object (glm tree) not returned; using party package [closed]

I'm trying to plot a glm tree using the package party. Per the reference guide, the default plot of the terminal node should be a spinogram as in this image but I ...
107 views

### Optimization program

I am trying to merge (and sum) $m$ time series into 2 groups. My goal is to minimize the standard deviation of the difference between the 2 groups. In other word, there can be an offset between the 2 ...
66 views

### Statistical test of fairness of partitioning

I have a question about a statistical test, but I lack the proper terminology so I'll try to explain in my own words. Say I have a collection of $N$ objects. The collection is partitioned into $p$ ...
98 views

### Variation partition analysis: morphology-climate correlation, controlling for geography

I'm working with three datasets with informations about individuals: morphological data, climatic data of the individual sampling site and geographical data. My objective is to assess whether ...
21 views

### Cluster/partition by time

I have a dataset of different events in time. I want to group/cluster/partition the data by datetime. A small example of the time of the data might be: [19-09-2018 12:00, 19-09-2018 12:01, 19-09-2018 ...
28 views

### Estimation of partitioning error in next-generation sequencing experiments

[Edited: explanation of the partitioning error] I would like to estimate how the initial number of molecules (or the level of gene expression) affects reproducibility between technical replicates of ...
634 views

### Adjusted Rand Index with different size inputs

Forgive me if my terminology is off, this concept is somewhat new to me. I'm trying to run an ARI calculation between two partitions which contain a different number of clusters, where each cluster ...
190 views

### Appropriate partition region in a decision tree?

I have the following partition region for a decision tree, but I do not think it is a valid region: $R=(\vec{X}=(X_1, X_2, ..., X_p)^T\in \mathbb{R}^p: X_1X_2 \geq 3)$ I feel that this is not a ...
309 views

### Predictors in random Forest

I am building a random forest to predict a binary variable y. I have several predictors named x1..n. One predictor, lets say x1, is a very strong predictor of y but only in some cases (see below) ...
59 views

### Estimates for k-d tree partition cells and for Voronoi partition cells

Let us have a $k$-d tree in a unit cube in ${\mathbb R}^k$. Let the points of the tree in the cube be i.i.d. with some density bounded away from 0. How can I estimate the volumes of partition ...
159 views

### How to partition set of items into subsets with similar mean, variance and number of elements - looking for some help

Short version: Given I have a list of 120 traits (happy, sad, etc.) evaluated on a number of scales ranging from -5 (for example highly negative) to +5 (for example highly positive), how can I group ...
462 views

### Possible explanations for Imputation before train-test split?

I'm working on a real world data set containing missing information. I understand imputing missing values before data partitioning can lead to leakage of information. I'm using this R package MissMech ...
71 views

### Clustering restaurant dishes

I have data about restaurants, their dishes and customer's orders. Then I want to cluster dishes in restaurant's menu by customer order. So clusters will be dishes in order and criteria of similarity ...
57 views

### Cluster non-vector data based on similarity rating

I have a list $S$ of $N$ non-vector elements (in this case words or concepts), and I'd like to partition them into k subsets of similar words. For each pair $(a,b)$ with $a,b \in S$, I have a number ...
53 views

### What statistical methods should i use to identify segments in a population according to a response variable?

Assuming I have $p$ uncorrelated explanatory variables $(X_i)_{i=1...p}$ and one binary response variable $Y$, such that : \log\left(\dfrac{P(Y=0|X)}{1-P(Y=0|X)}\right)=1_{\{X_1>0\}}\sum_{i=2}^p\...
37 views

### How is graph clustering performed? [closed]

When we do graph partitioning or graph clustering how does one give the graph as input? Is it by means of an adjacency matrix? What if the graph is very large?
537 views

### Is it compulsory to partition data in training and testing for time series modelling?

Is it compulsory to partition data in training and testing for time series modelling ? The reason for asking this question: Lets say training data= first 80% observation of actual time series data(...
48 views

### Smart sampling to learn a space partitioning

Given a compact feature space, I want to partition the space according to some classification. The first method that comes to mind is to use SVMs. For the classification I have in mind, the space ...
69 views

### Valid to supply unique features to each fold of k-fold Cross Validation?

I have a dataset of n = 55 samples with 300,000 - 600,000 features for each sample. I am trying to train a model (ksvm classification) to predict the class of each sample. N = 26 of the samples are ...
2k views

### Partitioning with cross validation

I am new to data analytics having only started exploring the field this week. I have downloaded KNIME and am working with a single dataset to try out different classification algorithms. I am ...
70 views

### How to combine partitioned dataset to single dataset?

I have been doing some Machine learning research on Microarray dataset. So, I downloaded breast cancer dataset from "http://datam.i2r.a-star.edu.sg/datasets/krbd/BreastCancer". The problem here is , ...
61 views

### How do loan companies set interest rate tiers?

What statistical or machine learning methods do companies like Lending Club use to segment their customer base into loan grades A1-G5? What would a reasonable partitioning method look like after ...
781 views

### Calculating covariances

Currently I am preparing for my exam. Therefore, we have a sample exam with the answers included. Unfortunately, even with the answers it not all makes sense to me. This is the question with the ...
111 views

### Partition Z-Scores Across Multiple Factors in R

Basic question Is there a way to create z-scores across multiple factors quickly? And how would you do this without creating multiple subsets and rebinding them? Details Basically I am comparing ...
616 views

### Lower bound for Adjusted Rand Index?

From the original paper, it's not clear whether the adjusted rand index has a lower bound. Does it? If so, what partition yields the bound? If now, how can I construct partitions with arbitrary low ...
119 views

### how do you calculate the number of combinations recursive partitioning uses in decision trees?

So I was wondering why random forests limit the levels of a variable to 32 levels, and I found an answer here : R's randomForest can not handle more than 32 levels. What is workaround? My ...
695 views

### Dividing a sample based on the value of y would be problematic?

Is it a well known fact in basic econometric that, if one divides a sample based on the value of Y (Dependent variable), it creates a number of problems? My dependent variable is financial rating ...
76 views

### Comparison of two different models usig the same CV partition

I am trying to compare two different models with cross validation and I am not sure whether I choosed an effective way to split my dataset into equal sized partitions. My data frame is called ...
86 views

### Assigning values to time dependent variables in Train/Test sets

In a dataset with history of customer visits, I have a variable denoting 'Days since last visit'. I have done a stratified train/test split of the data. How should I assign values to this variable? ...
3k views

### Dividing data into two groups as similar as possible

In this simplified example, I have five variables for each individual. The data is normalized, with mean 0 and sd 1 in each variable. I would like to divide this data into two groups with equal size ...
228 views

### Visualizing differences between alternative clusterings?

Does anyone know of a good way to visualize differences between alternative clusterings? I know there are some measures which can be used to quantify the similarity between two partitionings, e.g.: ...
3k views

### Extract the different components of variance in a linear mixed model in R

Consider a mixed model as follows. ...