Questions tagged [partitioning]

A partition is an assignment of every element of a set into 1 & only 1 subset w/ no empty subsets. A common instance of partitioning in statistics is the partitioning of sums of squares for F-tests.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Multicollinearity in polynomial regression in R

I fit a model to my data with the following formula after a stepwise selection (x1 to x5 stand for my variables): lm(formula = outcome ~ x1+ x3 + x4 + x2+ x5+ poly(x4, 2) + poly(x2, 2) + poly(x5, 2) + ...
user avatar
  • 1
0 votes
1 answer
33 views

Partitioning survival data into prognostic groups

I have survival data (n ~ 1000) with ~40% censoring and ~10% missing data. I want to partition the subjects into 4 prognostic groups based on survival. Misganow B and Vidyasagar M (Exploiting Ordinal ...
user avatar
1 vote
1 answer
15 views

How choose the number of classes of a series?

I am newbie in DS. I'm doing a choropleth map with votes of a senator in several cities. And I have two questions: 1 - How choose the correct number of classes to part the data? At the time im using ...
0 votes
0 answers
13 views

How can I disaggregate the impact of a group of variables using machine learning?

I have a problem where the target variable Y (continuous, values: 0-1) is controlled by large number of variables. These variables can be grouped by the nature of the data: ...
user avatar
1 vote
1 answer
39 views

fast distance metric between a new data entry and available clusters of data

Assume we divide a large data set D into m different partitions of data in a distributed learning case, using ...
user avatar
  • 125
2 votes
2 answers
92 views

Models for describing cluster change over time

Suppose I have a set of observations encoded as a matrix $X$ whose entries $X_{it} \in \mathcal{C} $ correspond to cluster labels of experimental unit $i$ with repeated measures at time $t$, where $\...
user avatar
  • 2,049
0 votes
0 answers
24 views

Hierarchical partitioning: negative values of independent contribution

I'm using the hier.part function in R to assess which variables explain the most variance. Therefore, I'm interested in the independent contribution of each predictor as a percentage of total ...
user avatar
0 votes
0 answers
10 views

Cluster permutation test explained in simple terms

I am working with a set of EEG time-frequency data (ie. activity across time) that requires some stats to identify time windows where activity surpasses a threshold. One such test that could be used ...
user avatar
1 vote
0 answers
27 views

Which type of sum of squares corresponds to the partition of variance formula for two-way ANOVA?

Does anyone have a good reference for the formula for type I and type III sum of squares in two-way ANOVA? I learned the following formula for partition of total sum of squares in two-way ANOVA. Is it ...
user avatar
  • 475
2 votes
0 answers
45 views

Partition a vector into equally sized groups

I have a vector of positive numbers x in R and I want to find a way to partition it into k groups so that all the groups have ...
user avatar
1 vote
1 answer
54 views

Variation partitioning using a large matrix as a predictor

I'm trying to understand the amount of variance explained in a univariate response using multiple community matrices as predictors. The problem is set up as such: I have the measured rate of a process ...
user avatar
0 votes
0 answers
27 views

n-way contingency tables comparison

I have a dataset containing various methods of splitting data into two groups. The true partition is not known, and I want to compare how similar the respective methods of partitioning are to each ...
user avatar
3 votes
1 answer
73 views

Should the median intuitively split a dataset into equally sized halves - and should two medians split into equally sized fourths?

I cannot decide if this question is patently silly or actually deep, so figured that makes it a perfect question for Cross Validated. We have a dataset with N ...
user avatar
1 vote
1 answer
17 views

How to check whether a partitioning is more balanced?

For example consider the following two partitions that add approximately to 100 percent. Partition 1 : 41% 36% 14% 5% 2% 1% <1% <1% Partition 2 : 82% 10% 4% 3% <1% <1% <1% <1% ...
user avatar
6 votes
1 answer
284 views

Sampling uniformly from the set of partitions of a set?

In this blogpost, the writer states "It’s easy to sample uniformly from the set of partitions of a set: you pick a number of bins using an appropriate exponential distribution, then randomly i.i....
user avatar
0 votes
0 answers
100 views

Clustering a distance matrix with k-medoids

For a symmetric distance matrix that I want to cluster, I performed several cluster algorithms: MDS into k-Means DBSCAN OPTICS k-Medoids (the one I'm having trouble with) Now, I would like to know ...
user avatar
1 vote
1 answer
14 views

Bayesian evaluation if partitioning is justified for a dataset

I'd like to comparare whether partitioning of a dataset is justified. The data is categorical with two levels and the fitted parameter is the prevalence of positives for a certain condition in each ...
user avatar
  • 2,313
4 votes
1 answer
28 views

Learn how to bisect 2d data to arrive at 1d relationships

I want to find a way to fit a series of curves throught 2d data. I have a hypothsis of how this can be done (I will explain it below). I would appreciate if anyone could suggest whether this technique ...
user avatar
  • 336
1 vote
1 answer
16 views

A line through the origin that bounds a proportion of points in a scatterplot

Suppose I have a scatterplot in a box with 0 < x < x1 and y1 < y < y2. Let 0 < prp < 1. Is there an efficient way to find a line that passes through the origin, and that has prp ...
user avatar
  • 113
18 votes
3 answers
2k views

Data partitioning for spatial data

I am constructing different configurations of a Random Forest in order to investigate the influence of well-design variables and location, on the first-year production volumes of shale oil wells, ...
user avatar
4 votes
1 answer
83 views

Exact distribution of number of transitions in a binary vector of length $n$

I am trying to derive an analytic solution for the exact distribution of the number of transitions in a binary vector of length $n$ conditional on the observed number of $1$'s and $0$'s and under the ...
user avatar
  • 1,029
0 votes
0 answers
333 views

variation partitioning with a GAMM model including an auto-correlation structure in R

I would like to undertake variation partitioning in a GAM framework in R, as described here: https://r.789695.n4.nabble.com/variance-explained-by-each-term-in-a-GAM-td836513.html However, my gam ...
user avatar
  • 43
2 votes
1 answer
97 views

Finding the probability of winning on or before n trials? [closed]

I have two teams A and B, the probability that A wins each hole is p, the probability that B wins each hole is q and the probability that neither wins is r. What is the probability of A winning on or ...
user avatar
2 votes
1 answer
149 views

In this concrete example of applying sklearn knn (with kd_tree) on Iris Data Set, how many partitions are there?

The k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input consists of the k closest training examples in the ...
user avatar
1 vote
0 answers
43 views

Recursive partitioning tree vs neural network model

I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression ...
user avatar
  • 31
1 vote
2 answers
593 views

Does the multinomial coefficient create combinations or permutations?

I am looking at this formula I just want to make sure I get it. For example, if I want to split a fifty-two card deck into all the possible combinations of four, thirteen card hands, then I get: $$\...
user avatar
  • 295
0 votes
1 answer
244 views

default plot for mob object (glm tree) not returned; using party package [closed]

I'm trying to plot a glm tree using the package party. Per the reference guide, the default plot of the terminal node should be a spinogram as in this image but I ...
user avatar
  • 89
2 votes
1 answer
126 views

Optimization program to allocate time series in 2 groups

I am trying to dispatch $m$ time series into 2 groups. Then I want to sum the value for each time steps, to obtain a sum per group. My goal is to minimize the standard deviation of the difference ...
user avatar
  • 1,469
0 votes
1 answer
67 views

Statistical test of fairness of partitioning

I have a question about a statistical test, but I lack the proper terminology so I'll try to explain in my own words. Say I have a collection of $N$ objects. The collection is partitioned into $p$ ...
user avatar
1 vote
0 answers
128 views

Variation partition analysis: morphology-climate correlation, controlling for geography

I'm working with three datasets with informations about individuals: morphological data, climatic data of the individual sampling site and geographical data. My objective is to assess whether ...
user avatar
1 vote
0 answers
24 views

Cluster/partition by time

I have a dataset of different events in time. I want to group/cluster/partition the data by datetime. A small example of the time of the data might be: [19-09-2018 12:00, 19-09-2018 12:01, 19-09-2018 ...
user avatar
  • 11
0 votes
1 answer
30 views

Estimation of partitioning error in next-generation sequencing experiments

[Edited: explanation of the partitioning error] I would like to estimate how the initial number of molecules (or the level of gene expression) affects reproducibility between technical replicates of ...
user avatar
0 votes
1 answer
1k views

Adjusted Rand Index with different size inputs

Forgive me if my terminology is off, this concept is somewhat new to me. I'm trying to run an ARI calculation between two partitions which contain a different number of clusters, where each cluster ...
user avatar
2 votes
1 answer
220 views

Appropriate partition region in a decision tree?

I have the following partition region for a decision tree, but I do not think it is a valid region: $R=(\vec{X}=(X_1, X_2, ..., X_p)^T\in \mathbb{R}^p: X_1X_2 \geq 3)$ I feel that this is not a ...
user avatar
4 votes
2 answers
371 views

Predictors in random Forest

I am building a random forest to predict a binary variable y. I have several predictors named x1..n. One predictor, lets say x1, is a very strong predictor of y but only in some cases (see below) ...
user avatar
  • 173
2 votes
0 answers
62 views

Estimates for k-d tree partition cells and for Voronoi partition cells

Let us have a $k$-d tree in a unit cube in ${\mathbb R}^k$. Let the points of the tree in the cube be i.i.d. with some density bounded away from 0. How can I estimate the volumes of partition ...
user avatar
  • 883
2 votes
1 answer
195 views

How to partition set of items into subsets with similar mean, variance and number of elements - looking for some help

Short version: Given I have a list of 120 traits (happy, sad, etc.) evaluated on a number of scales ranging from -5 (for example highly negative) to +5 (for example highly positive), how can I group ...
user avatar
  • 507
2 votes
1 answer
604 views

Possible explanations for Imputation before train-test split?

I'm working on a real world data set containing missing information. I understand imputing missing values before data partitioning can lead to leakage of information. I'm using this R package MissMech ...
user avatar
  • 33
-1 votes
1 answer
80 views

Clustering restaurant dishes

I have data about restaurants, their dishes and customer's orders. Then I want to cluster dishes in restaurant's menu by customer order. So clusters will be dishes in order and criteria of similarity ...
user avatar
1 vote
2 answers
92 views

Cluster non-vector data based on similarity rating

I have a list $S$ of $N$ non-vector elements (in this case words or concepts), and I'd like to partition them into k subsets of similar words. For each pair $(a,b)$ with $a,b \in S$, I have a number ...
user avatar
0 votes
1 answer
56 views

What statistical methods should i use to identify segments in a population according to a response variable?

Assuming I have $p$ uncorrelated explanatory variables $(X_i)_{i=1...p}$ and one binary response variable $Y$, such that : $$\log\left(\dfrac{P(Y=0|X)}{1-P(Y=0|X)}\right)=1_{\{X_1>0\}}\sum_{i=2}^p\...
user avatar
  • 11
-1 votes
1 answer
38 views

How is graph clustering performed? [closed]

When we do graph partitioning or graph clustering how does one give the graph as input? Is it by means of an adjacency matrix? What if the graph is very large?
user avatar
2 votes
1 answer
864 views

Is it compulsory to partition data in training and testing for time series modelling?

Is it compulsory to partition data in training and testing for time series modelling ? The reason for asking this question: Lets say training data= first 80% observation of actual time series data(...
user avatar
1 vote
0 answers
50 views

Smart sampling to learn a space partitioning

Given a compact feature space, I want to partition the space according to some classification. The first method that comes to mind is to use SVMs. For the classification I have in mind, the space ...
user avatar
  • 111
0 votes
0 answers
69 views

Valid to supply unique features to each fold of k-fold Cross Validation?

I have a dataset of n = 55 samples with 300,000 - 600,000 features for each sample. I am trying to train a model (ksvm classification) to predict the class of each sample. N = 26 of the samples are ...
user avatar
  • 225
1 vote
1 answer
2k views

Partitioning with cross validation?

I am new to data analytics having only started exploring the field this week. I have downloaded KNIME and am working with a single dataset to try out different classification algorithms. I am ...
user avatar
  • 11
0 votes
2 answers
87 views

How to combine partitioned dataset to single dataset?

I have been doing some Machine learning research on Microarray dataset. So, I downloaded breast cancer dataset from "http://datam.i2r.a-star.edu.sg/datasets/krbd/BreastCancer". The problem here is , ...
user avatar
  • 123
2 votes
0 answers
63 views

How do loan companies set interest rate tiers?

What statistical or machine learning methods do companies like Lending Club use to segment their customer base into loan grades A1-G5? What would a reasonable partitioning method look like after ...
user avatar
  • 21
2 votes
1 answer
816 views

Calculating covariances

Currently I am preparing for my exam. Therefore, we have a sample exam with the answers included. Unfortunately, even with the answers it not all makes sense to me. This is the question with the ...
user avatar
0 votes
0 answers
113 views

Partition Z-Scores Across Multiple Factors in R

Basic question Is there a way to create z-scores across multiple factors quickly? And how would you do this without creating multiple subsets and rebinding them? Details Basically I am comparing ...
user avatar
  • 78