Questions tagged [partitioning]

A partition is an assignment of every element of a set into 1 & only 1 subset w/ no empty subsets. A common instance of partitioning in statistics is the partitioning of sums of squares for F-tests.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Clustering a distance matrix with k-medoids

For a symmetric distance matrix that I want to cluster, I performed several cluster algorithms: MDS into k-Means DBSCAN OPTICS k-Medoids (the one I'm having trouble with) Now, I would like to know ...
0
votes
0answers
15 views

Use inference on the whole data as a prior to regularize estimates in a subgroup

Suppose that the probability of a certain condition in the data $D$ is $\theta$ which can be estimated by updating a Beta prior as $Pr(\theta|s, N) = Beta(\alpha + s, \beta + N - s)$ with $s$ being ...
1
vote
1answer
11 views

Bayesian evaluation if partitioning is justified for a dataset

I'd like to comparare whether partitioning of a dataset is justified. The data is categorical with two levels and the fitted parameter is the prevalence of positives for a certain condition in each ...
0
votes
0answers
9 views

How can I apply partitioning chi square test?

I have a table that has 4 groups and 5 comparisons. Groups are: monthly, several time annually, rarely and never. Comparisons are: stage 0, 1, 2, 3, 4. Chi square test were performed. There is a ...
0
votes
0answers
8 views

Representing partitioned N-dimensional input data

I am working on N-dimensional inputs and single dimensional response data sets. I want to get local picture of slopes in my response and hence, I am dividing my input space into local regions by ...
3
votes
1answer
22 views

Learn how to bisect 2d data to arrive at 1d relationships

I want to find a way to fit a series of curves throught 2d data. I have a hypothsis of how this can be done (I will explain it below). I would appreciate if anyone could suggest whether this technique ...
0
votes
0answers
5 views

When/where do I correlation analysis during cross validation using training/test (holdout) partitions?

I'm doing Multiple Regression Right now I'm using a training partition to do [partial] correlation analysis. Then once done I split the data up into folds and do backwards step cross validation and ...
0
votes
0answers
34 views

If likelihood is statistically sufficient, how could some cases have no sufficient statistics?

Almost all elementary texts clarify that in some cases minimal sufficient statistics might not exist. However, it seems that the likelihood itself induces a partition that essentially provides a ...
1
vote
1answer
16 views

A line through the origin that bounds a proportion of points in a scatterplot

Suppose I have a scatterplot in a box with 0 < x < x1 and y1 < y < y2. Let 0 < prp < 1. Is there an efficient way to find a line that passes through the origin, and that has prp ...
17
votes
3answers
926 views

Data partitioning for spatial data

I am constructing different configurations of a Random Forest in order to investigate the influence of well-design variables and location, on the first-year production volumes of shale oil wells, ...
3
votes
1answer
45 views

Exact distribution of number of transitions in a binary vector of length $n$

I am trying to derive an analytic solution for the exact distribution of the number of transitions in a binary vector of length $n$ conditional on the observed number of $1$'s and $0$'s and under the ...
0
votes
0answers
72 views

variation partitioning with a GAMM model including an auto-correlation structure in R

I would like to undertake variation partitioning in a GAM framework in R, as described here: https://r.789695.n4.nabble.com/variance-explained-by-each-term-in-a-GAM-td836513.html However, my gam ...
2
votes
1answer
49 views

Finding the probability of winning on or before n trials? [closed]

I have two teams A and B, the probability that A wins each hole is p, the probability that B wins each hole is q and the probability that neither wins is r. What is the probability of A winning on or ...
2
votes
1answer
37 views

In this concrete example of applying sklearn knn (with kd_tree) on Iris Data Set, how many partitions are there?

The k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. In both cases, the input consists of the k closest training examples in the ...
1
vote
0answers
27 views

Recursive partitioning tree vs neural network model

I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression ...
0
votes
1answer
104 views

Does the multinomial coefficient create combinations or permutations?

I am looking at this formula I just want to make sure I get it. For example, if I want to split a fifty-two card deck into all the possible combinations of four, thirteen card hands, then I get: 52!...
0
votes
1answer
157 views

default plot for mob object (glm tree) not returned; using party package [closed]

I'm trying to plot a glm tree using the package party. Per the reference guide, the default plot of the terminal node should be a spinogram as in this image but I ...
1
vote
1answer
107 views

Optimization program

I am trying to merge (and sum) $m$ time series into 2 groups. My goal is to minimize the standard deviation of the difference between the 2 groups. In other word, there can be an offset between the 2 ...
0
votes
1answer
66 views

Statistical test of fairness of partitioning

I have a question about a statistical test, but I lack the proper terminology so I'll try to explain in my own words. Say I have a collection of $N$ objects. The collection is partitioned into $p$ ...
1
vote
0answers
98 views

Variation partition analysis: morphology-climate correlation, controlling for geography

I'm working with three datasets with informations about individuals: morphological data, climatic data of the individual sampling site and geographical data. My objective is to assess whether ...
1
vote
0answers
21 views

Cluster/partition by time

I have a dataset of different events in time. I want to group/cluster/partition the data by datetime. A small example of the time of the data might be: [19-09-2018 12:00, 19-09-2018 12:01, 19-09-2018 ...
0
votes
1answer
28 views

Estimation of partitioning error in next-generation sequencing experiments

[Edited: explanation of the partitioning error] I would like to estimate how the initial number of molecules (or the level of gene expression) affects reproducibility between technical replicates of ...
0
votes
1answer
634 views

Adjusted Rand Index with different size inputs

Forgive me if my terminology is off, this concept is somewhat new to me. I'm trying to run an ARI calculation between two partitions which contain a different number of clusters, where each cluster ...
2
votes
1answer
190 views

Appropriate partition region in a decision tree?

I have the following partition region for a decision tree, but I do not think it is a valid region: $R=(\vec{X}=(X_1, X_2, ..., X_p)^T\in \mathbb{R}^p: X_1X_2 \geq 3)$ I feel that this is not a ...
4
votes
2answers
309 views

Predictors in random Forest

I am building a random forest to predict a binary variable y. I have several predictors named x1..n. One predictor, lets say x1, is a very strong predictor of y but only in some cases (see below) ...
2
votes
0answers
59 views

Estimates for k-d tree partition cells and for Voronoi partition cells

Let us have a $k$-d tree in a unit cube in ${\mathbb R}^k$. Let the points of the tree in the cube be i.i.d. with some density bounded away from 0. How can I estimate the volumes of partition ...
0
votes
1answer
159 views

How to partition set of items into subsets with similar mean, variance and number of elements - looking for some help

Short version: Given I have a list of 120 traits (happy, sad, etc.) evaluated on a number of scales ranging from -5 (for example highly negative) to +5 (for example highly positive), how can I group ...
1
vote
1answer
462 views

Possible explanations for Imputation before train-test split?

I'm working on a real world data set containing missing information. I understand imputing missing values before data partitioning can lead to leakage of information. I'm using this R package MissMech ...
-1
votes
1answer
71 views

Clustering restaurant dishes

I have data about restaurants, their dishes and customer's orders. Then I want to cluster dishes in restaurant's menu by customer order. So clusters will be dishes in order and criteria of similarity ...
1
vote
2answers
57 views

Cluster non-vector data based on similarity rating

I have a list $S$ of $N$ non-vector elements (in this case words or concepts), and I'd like to partition them into k subsets of similar words. For each pair $(a,b)$ with $a,b \in S$, I have a number ...
0
votes
1answer
53 views

What statistical methods should i use to identify segments in a population according to a response variable?

Assuming I have $p$ uncorrelated explanatory variables $(X_i)_{i=1...p}$ and one binary response variable $Y$, such that : $$\log\left(\dfrac{P(Y=0|X)}{1-P(Y=0|X)}\right)=1_{\{X_1>0\}}\sum_{i=2}^p\...
-1
votes
1answer
37 views

How is graph clustering performed? [closed]

When we do graph partitioning or graph clustering how does one give the graph as input? Is it by means of an adjacency matrix? What if the graph is very large?
2
votes
1answer
537 views

Is it compulsory to partition data in training and testing for time series modelling?

Is it compulsory to partition data in training and testing for time series modelling ? The reason for asking this question: Lets say training data= first 80% observation of actual time series data(...
1
vote
0answers
48 views

Smart sampling to learn a space partitioning

Given a compact feature space, I want to partition the space according to some classification. The first method that comes to mind is to use SVMs. For the classification I have in mind, the space ...
0
votes
0answers
69 views

Valid to supply unique features to each fold of k-fold Cross Validation?

I have a dataset of n = 55 samples with 300,000 - 600,000 features for each sample. I am trying to train a model (ksvm classification) to predict the class of each sample. N = 26 of the samples are ...
0
votes
1answer
2k views

Partitioning with cross validation

I am new to data analytics having only started exploring the field this week. I have downloaded KNIME and am working with a single dataset to try out different classification algorithms. I am ...
0
votes
2answers
70 views

How to combine partitioned dataset to single dataset?

I have been doing some Machine learning research on Microarray dataset. So, I downloaded breast cancer dataset from "http://datam.i2r.a-star.edu.sg/datasets/krbd/BreastCancer". The problem here is , ...
2
votes
0answers
61 views

How do loan companies set interest rate tiers?

What statistical or machine learning methods do companies like Lending Club use to segment their customer base into loan grades A1-G5? What would a reasonable partitioning method look like after ...
2
votes
1answer
781 views

Calculating covariances

Currently I am preparing for my exam. Therefore, we have a sample exam with the answers included. Unfortunately, even with the answers it not all makes sense to me. This is the question with the ...
0
votes
0answers
111 views

Partition Z-Scores Across Multiple Factors in R

Basic question Is there a way to create z-scores across multiple factors quickly? And how would you do this without creating multiple subsets and rebinding them? Details Basically I am comparing ...
2
votes
2answers
616 views

Lower bound for Adjusted Rand Index?

From the original paper, it's not clear whether the adjusted rand index has a lower bound. Does it? If so, what partition yields the bound? If now, how can I construct partitions with arbitrary low ...
0
votes
0answers
119 views

how do you calculate the number of combinations recursive partitioning uses in decision trees?

So I was wondering why random forests limit the levels of a variable to 32 levels, and I found an answer here : R's randomForest can not handle more than 32 levels. What is workaround? My ...
2
votes
2answers
695 views

Dividing a sample based on the value of y would be problematic?

Is it a well known fact in basic econometric that, if one divides a sample based on the value of Y (Dependent variable), it creates a number of problems? My dependent variable is financial rating ...
1
vote
0answers
76 views

Comparison of two different models usig the same CV partition

I am trying to compare two different models with cross validation and I am not sure whether I choosed an effective way to split my dataset into equal sized partitions. My data frame is called ...
1
vote
0answers
86 views

Assigning values to time dependent variables in Train/Test sets

In a dataset with history of customer visits, I have a variable denoting 'Days since last visit'. I have done a stratified train/test split of the data. How should I assign values to this variable? ...
2
votes
1answer
3k views

Dividing data into two groups as similar as possible

In this simplified example, I have five variables for each individual. The data is normalized, with mean 0 and sd 1 in each variable. I would like to divide this data into two groups with equal size ...
5
votes
1answer
228 views

Visualizing differences between alternative clusterings?

Does anyone know of a good way to visualize differences between alternative clusterings? I know there are some measures which can be used to quantify the similarity between two partitionings, e.g.: ...
0
votes
1answer
3k views

Extract the different components of variance in a linear mixed model in R

Consider a mixed model as follows. ...
2
votes
3answers
2k views

Repeatedly split data in training (0.75) and test (0.25) for cross validation

What kind of cross validation is it called when we randomly split the data into 0.75 training and 0.25 test data set. And this split is done 1000 times.
1
vote
0answers
395 views

How to calculate VPC in glmer?

For a specific analysis I want to calculate the variance partition coefficient (VPC). I am using the following formula: ...