Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [partitioning]

A partition is an assignment of every element of a set into 1 & only 1 subset w/ no empty subsets. A common instance of partitioning in statistics is the partitioning of sums of squares for F-tests.

1
vote
0answers
5 views

Recursive partitioning tree vs neural network model

I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression ...
0
votes
1answer
11 views

Does the multinomial coefficient create combinations or permutations?

I am looking at this formula I just want to make sure I get it. For example, if I want to split a fifty-two card deck into all the possible combinations of four, thirteen card hands, then I get: 52!...
0
votes
0answers
44 views

Random partition techniques for lasso and elastic net

I think this is the correct place for this question. I have implemented lasso, elastic net and a different estimator on a real data set. I used 10-fold cross-validation (CV) one time without ...
0
votes
1answer
55 views

default plot for mob object (glm tree) not returned; using party package [closed]

I'm trying to plot a glm tree using the package party. Per the reference guide, the default plot of the terminal node should be a spinogram as in this image but I ...
0
votes
1answer
61 views

Optimization program

I am trying to merge (and sum) $m$ time series into 2 groups. My goal is to minimize the standard deviation of the difference between the 2 groups. In other word, there can be an offset between the 2 ...
0
votes
0answers
16 views

AUC/R^2 score strictly lower on partitioned dataset than total dataset

Suppose I've fit a logistic regression to my data and I calculate the AUC. I then partition my dataset according to one of the explanatory variables (say those rows where the variable value is greater/...
0
votes
1answer
59 views

Statistical test of fairness of partitioning

I have a question about a statistical test, but I lack the proper terminology so I'll try to explain in my own words. Say I have a collection of $N$ objects. The collection is partitioned into $p$ ...
1
vote
0answers
71 views

Variation partition analysis: morphology-climate correlation, controlling for geography

I'm working with three datasets with informations about individuals: morphological data, climatic data of the individual sampling site and geographical data. My objective is to assess whether ...
1
vote
0answers
13 views

Cluster/partition by time

I have a dataset of different events in time. I want to group/cluster/partition the data by datetime. A small example of the time of the data might be: [19-09-2018 12:00, 19-09-2018 12:01, 19-09-2018 ...
0
votes
1answer
26 views

Estimation of partitioning error in next-generation sequencing experiments

[Edited: explanation of the partitioning error] I would like to estimate how the initial number of molecules (or the level of gene expression) affects reproducibility between technical replicates of ...
0
votes
1answer
257 views

Adjusted Rand Index with different size inputs

Forgive me if my terminology is off, this concept is somewhat new to me. I'm trying to run an ARI calculation between two partitions which contain a different number of clusters, where each cluster ...
0
votes
0answers
9 views

Appropriateness of explicitly modelling sums in hierarchical models

My lab is interested in morphological measures that have a partition structure. A simple example that has the same property would be studying effects on arm lengths. Each person has an armspan which ...
2
votes
1answer
131 views

Appropriate partition region in a decision tree?

I have the following partition region for a decision tree, but I do not think it is a valid region: $R=(\vec{X}=(X_1, X_2, ..., X_p)^T\in \mathbb{R}^p: X_1X_2 \geq 3)$ I feel that this is not a ...
4
votes
2answers
210 views

Predictors in random Forest

I am building a random forest to predict a binary variable y. I have several predictors named x1..n. One predictor, lets say x1, is a very strong predictor of y but only in some cases (see below) ...
2
votes
0answers
56 views

Estimates for k-d tree partition cells and for Voronoi partition cells

Let us have a $k$-d tree in a unit cube in ${\mathbb R}^k$. Let the points of the tree in the cube be i.i.d. with some density bounded away from 0. How can I estimate the volumes of partition ...
0
votes
1answer
135 views

How to partition set of items into subsets with similar mean, variance and number of elements - looking for some help

Short version: Given I have a list of 120 traits (happy, sad, etc.) evaluated on a number of scales ranging from -5 (for example highly negative) to +5 (for example highly positive), how can I group ...
1
vote
1answer
222 views

Possible explanations for Imputation before train-test split?

I'm working on a real world data set containing missing information. I understand imputing missing values before data partitioning can lead to leakage of information. I'm using this R package MissMech ...
-1
votes
1answer
47 views

Clustering restaurant dishes

I have data about restaurants, their dishes and customer's orders. Then I want to cluster dishes in restaurant's menu by customer order. So clusters will be dishes in order and criteria of similarity ...
0
votes
2answers
47 views

Cluster non-vector data based on similarity rating

I have a list $S$ of $N$ non-vector elements (in this case words or concepts), and I'd like to partition them into k subsets of similar words. For each pair $(a,b)$ with $a,b \in S$, I have a number ...
0
votes
1answer
49 views

What statistical methods should i use to identify segments in a population according to a response variable?

Assuming I have $p$ uncorrelated explanatory variables $(X_i)_{i=1...p}$ and one binary response variable $Y$, such that : $$\log\left(\dfrac{P(Y=0|X)}{1-P(Y=0|X)}\right)=1_{\{X_1>0\}}\sum_{i=2}^p\...
-1
votes
1answer
35 views

How is graph clustering performed? [closed]

When we do graph partitioning or graph clustering how does one give the graph as input? Is it by means of an adjacency matrix? What if the graph is very large?
2
votes
1answer
286 views

Is it compulsory to partition data in training and testing for time series modelling?

Is it compulsory to partition data in training and testing for time series modelling ? The reason for asking this question: Lets say training data= first 80% observation of actual time series data(...
1
vote
0answers
47 views

Smart sampling to learn a space partitioning

Given a compact feature space, I want to partition the space according to some classification. The first method that comes to mind is to use SVMs. For the classification I have in mind, the space ...
0
votes
0answers
57 views

Valid to supply unique features to each fold of k-fold Cross Validation?

I have a dataset of n = 55 samples with 300,000 - 600,000 features for each sample. I am trying to train a model (ksvm classification) to predict the class of each sample. N = 26 of the samples are ...
0
votes
1answer
1k views

Partitioning with cross validation

I am new to data analytics having only started exploring the field this week. I have downloaded KNIME and am working with a single dataset to try out different classification algorithms. I am ...
0
votes
2answers
52 views

How to combine partitioned dataset to single dataset?

I have been doing some Machine learning research on Microarray dataset. So, I downloaded breast cancer dataset from "http://datam.i2r.a-star.edu.sg/datasets/krbd/BreastCancer". The problem here is , ...
2
votes
0answers
56 views

How do loan companies set interest rate tiers?

What statistical or machine learning methods do companies like Lending Club use to segment their customer base into loan grades A1-G5? What would a reasonable partitioning method look like after ...
2
votes
1answer
588 views

Calculating covariances

Currently I am preparing for my exam. Therefore, we have a sample exam with the answers included. Unfortunately, even with the answers it not all makes sense to me. This is the question with the ...
0
votes
0answers
109 views

Partition Z-Scores Across Multiple Factors in R

Basic question Is there a way to create z-scores across multiple factors quickly? And how would you do this without creating multiple subsets and rebinding them? Details Basically I am comparing ...
1
vote
2answers
452 views

Lower bound for Adjusted Rand Index?

From the original paper, it's not clear whether the adjusted rand index has a lower bound. Does it? If so, what partition yields the bound? If now, how can I construct partitions with arbitrary low ...
0
votes
0answers
67 views

how do you calculate the number of combinations recursive partitioning uses in decision trees?

So I was wondering why random forests limit the levels of a variable to 32 levels, and I found an answer here : R's randomForest can not handle more than 32 levels. What is workaround? My ...
2
votes
2answers
452 views

Dividing a sample based on the value of y would be problematic?

Is it a well known fact in basic econometric that, if one divides a sample based on the value of Y (Dependent variable), it creates a number of problems? My dependent variable is financial rating ...
1
vote
0answers
62 views

Comparison of two different models usig the same CV partition

I am trying to compare two different models with cross validation and I am not sure whether I choosed an effective way to split my dataset into equal sized partitions. My data frame is called ...
1
vote
0answers
67 views

Assigning values to time dependent variables in Train/Test sets

In a dataset with history of customer visits, I have a variable denoting 'Days since last visit'. I have done a stratified train/test split of the data. How should I assign values to this variable? ...
2
votes
1answer
1k views

Dividing data into two groups as similar as possible

In this simplified example, I have five variables for each individual. The data is normalized, with mean 0 and sd 1 in each variable. I would like to divide this data into two groups with equal size ...
5
votes
1answer
146 views

Visualizing differences between alternative clusterings?

Does anyone know of a good way to visualize differences between alternative clusterings? I know there are some measures which can be used to quantify the similarity between two partitionings, e.g.: ...
0
votes
1answer
2k views
2
votes
3answers
1k views

Repeatedly split data in training (0.75) and test (0.25) for cross validation

What kind of cross validation is it called when we randomly split the data into 0.75 training and 0.25 test data set. And this split is done 1000 times.
1
vote
0answers
221 views

How to calculate VPC in glmer?

For a specific analysis I want to calculate the variance partition coefficient (VPC). I am using the following formula: ...
1
vote
1answer
362 views

Random Forests with modified partitioning criteria

Here is the context of my question : I'm doing binary classification with unbalanced classes. The measure of performance I'd like to maximise is a modified F-measure : $$ F_{\alpha} = \frac{1}{\frac{\...
3
votes
2answers
205 views

Cluster real numbers

I have a set of precise measurements, and what I want to do is count the frequency (how many time it appears) for each value. The problem is that these are very precise measurements and with a naive ...
0
votes
1answer
64 views

How to choose the relative sizes of training and validation sets?

When I work with the methods of data mining, the data is split in training and validations data samples (and sometimes test). I know training + validation = 100%. Which criteria can I use to find a ...
4
votes
3answers
2k views

How to build a decision tree with a constraint on sensitivity?

I am trying to develop a classification model on a sample of people which will discriminate between "Type A" and "Not-Type A" people. Due to external factors, the minimum sensitivity for this ...
0
votes
1answer
50 views

Least Squared Regressions on Partitioned Data

Say I have 800 (X,Y) data points, and I do a LSQ fit and get y = mx+b Then I think to myself, of the 800 data points, 500 are males and 300 are females, so I ...
1
vote
1answer
2k views

What are the differences between Lloyd's, MacQueen's and Hartigan's algorithms for K-Means?

There are three distinct algorithms for the K-Means function in R. These are: Lloyd's MacQueen's Hartigan's I believe I understand how Lloyd's works. 1. The cluster centers are chosen. 2. Points are ...
7
votes
0answers
501 views

What approaches use multiple eigenvectors in graph spectral clustering?

Background: In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by ...
2
votes
0answers
671 views

Is K-Medoids really better at dealing with outliers than K-Means? (with example showing the contrary)

K-Medoids and K-Means are two popular methods of partitional clustering. The consensus is that K-Medoids is better at clustering data when there are outliers (source). This is because it chooses data ...
4
votes
0answers
58 views

Efficient ways to partition rows of augmented design matrix $[X|y]$ into subsets with similar regression results?

Imagine I have $n$ observations on a regression model; are there any reasonably efficient methods for partitioning that into two (or more) roughly equally sized groups which almost reproduce the ...
4
votes
0answers
178 views

Catalan numbers and gambler's ruin paths

Edit I've edited this question several times, and in the end I essentially answered my own question. No need to answer it yourself. I made it slightly less long and rambling, in case anyone else is ...
1
vote
0answers
90 views

Comparing two graphs/markov chains by comparing their clusters

I have an undirected graph representation of my system (a dynamical system), i.e. I have some labelled nodes and bi-directional edge weights, so everything is in a Markov matrix form. Now I can form ...