A partition is an assignment of every element of a set into 1 & only 1 subset w/ no empty subsets. A common instance of partitioning in statistics is the partitioning of sums of squares for F-tests.

learn more… | top users | synonyms

0
votes
2answers
36 views
0
votes
0answers
4 views

Creating data partition in R [migrated]

When creating data partition 75% training and 25% test, we use: inTrain<- createDataPartition(y=spam$type,p=0.75, list=FALSE) Note: dataset is named "spam" and target variable is named "type" My ...
0
votes
0answers
6 views

Hard partitioning of the association matrix

I obtain a co-association matrix $n \times n$ that corresponds to the maximum likelihood estimate of the probability of pairs of variables being in the same cluster. Further suppose that there are $k$ ...
1
vote
3answers
62 views

Repeatedly split data in training (0.75) and test (0.25) for cross validation

What kind of cross validation is it called when we randomly split the data into 0.75 training and 0.25 test data set. And this split is done 1000 times.
1
vote
0answers
20 views

How to calculate VPC in glmer?

For a specific analysis I want to calculate the variance partition coefficient (VPC). I am using the following formula: ...
0
votes
0answers
13 views

Incorporating shifting spatial autocorrelation into a GLMM

So I'm examining a series of sites across a landscape for how wildlife use of these sites changes following treatment (reclamation). Treatment of these sites randomly took place over three years, and ...
0
votes
0answers
10 views

Difference between adversarial, random and stochastic ordering in data streams

Kindly explain the difference between adversarial, random and stochastic ordering in data streams in layman terms
1
vote
1answer
70 views

How to partition leave-one-subject-out (not leave-one-example-out) cross-validation in MATLAB?

I am currently extracting 16 features from 7 samples all of different length. Now I would like to apply the data using multiple classification algorithms with cross validation. I already done this by ...
0
votes
1answer
30 views

Random Forests with modified partitioning criteria

Here is the context of my question : I'm doing binary classification with unbalanced classes. The measure of performance I'd like to maximise is a modified F-measure : $$ F_{\alpha} = \frac{1}{\frac{\...
3
votes
2answers
48 views

Cluster real numbers

I have a set of precise measurements, and what I want to do is count the frequency (how many time it appears) for each value. The problem is that these are very precise measurements and with a naive ...
0
votes
1answer
33 views

How to choose the relative sizes of training and validation sets?

When I work with the methods of data mining, the data is split in training and validations data samples (and sometimes test). I know training + validation = 100%. Which criteria can I use to find a ...
3
votes
2answers
168 views

How to build a decision tree with a constraint on sensitivity?

I am trying to develop a classification model on a sample of people which will discriminate between "Type A" and "Not-Type A" people. Due to external factors, the minimum sensitivity for this ...
0
votes
0answers
12 views

How to compare multiple non-hierarchical classications of the same dataset

I'm a taxonomist working on an identification consistency project in which a couple dozen researchers were asked to manually classify the same set of images into like groups. Each classification will ...
0
votes
1answer
43 views

Least Squared Regressions on Partitioned Data

Say I have 800 (X,Y) data points, and I do a LSQ fit and get y = mx+b Then I think to myself, of the 800 data points, 500 are males and 300 are females, so I ...
0
votes
0answers
172 views

What are the differences between Lloyd's, MacQueen's and Hartigan's algorithms for K-Means?

There are three distinct algorithms for the K-Means function in R. These are: Lloyd's MacQueen's Hartigan's I believe I understand how Lloyd's works. 1. The cluster centers are chosen. 2. Points ...
5
votes
0answers
157 views

Multiple eigenvectors in graph spectral clustering

In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by eigenvector of a ...
0
votes
0answers
7 views

Comparing groups that themselves are calculated from means

Context: I am calculating net energy benefit from a series of energy costs and gross energy benefits. For example, I have 3 measurements of gross benefit, replicated 4 times, for each of 4 ...
1
vote
0answers
39 views

Is K-Medoids really better at dealing with outliers than K-Means? (with example showing the contrary)

K-Medoids and K-Means are two popular methods of partitional clustering. The consensus is that K-Medoids is better at clustering data when there are outliers (source). This is because it chooses data ...
3
votes
0answers
17 views

Efficient ways to partition rows of augmented design matrix $[X|y]$ into subsets with similar regression results?

Imagine I have $n$ observations on a regression model; are there any reasonably efficient methods for partitioning that into two (or more) roughly equally sized groups which almost reproduce the ...
4
votes
0answers
99 views

Catalan numbers and gambler's ruin paths

Edit I've edited this question several times, and in the end I essentially answered my own question. No need to answer it yourself. I made it slightly less long and rambling, in case anyone else is ...
1
vote
0answers
38 views

Comparing two graphs/markov chains by comparing their clusters

I have an undirected graph representation of my system (a dynamical system), i.e. I have some labelled nodes and bi-directional edge weights, so everything is in a Markov matrix form. Now I can form ...
0
votes
0answers
17 views

Structural partition for cross validation

Can the partitioning of data for cross validation be used structurally to assess the transfer ability of an effect of one factor across another factor? A MANOVA performed on multivariate data reveals ...
-1
votes
1answer
23 views

Clustering a list of restaurant dishes

If I have a large list of restaurant dishes that all have the same cuisine... (Pulled Pork, BBQ chicken, 1/2 Ribs, Pork Sliders, Slow Smoked Pork, Full Chicken Special....) What would be a good ...
4
votes
1answer
57 views

Partition data into two sets such that the difference of their variance is minimal

Suppose there are $n$ data values $x_1<x_2<\ldots<x_{n-1}<x_n$,and I've found a partition number $k$, such that $$ \left|\frac{1}{k}\sum_{i=1}^k(x_i-\hat{\mu_k})^2-\frac{1}{n-k}\sum_{j=k+1}...
1
vote
0answers
93 views

How to partition $R^2$ among predictors in multiple regression with interaction terms in R

Say I have some predictors, and I know how they affect some dependent variable: ...
3
votes
2answers
145 views

Representative elements of a set

I'm looking for the technical name of the following problem. It sounds like a standard machine learning technique, but I'm not familiar with the field, and can't seem to find it. Let's say that we ...
2
votes
1answer
63 views

Visualise variance partitioning

I'm interested in visualising variance partitioning in the context of linear models. Say you run an linear regression that predicts peoples weight based on their height and age. How can you visualise ...
0
votes
1answer
145 views

K-Medoids swapping inside clusters

I'm a bit confused with concept of K-medoids. It seems that original algorithm (PAM) describes that swap step should be performed by swaping only one of the medoids with one non-medoid point from ...
0
votes
3answers
485 views

Data partitioning according two variables

I am working with the following dataset: ...
4
votes
1answer
87 views

Show that $\sqrt{ESS} \leq \sqrt{ESS_{A}}+\sqrt{ESS_{\bar{A}}}$ where ESS=Explained sum of squares

Suppose we have a dependent variable $Y$ with mean zero and set of regressors which we divide into two sets, $A$ and $\bar{A}$. Let $ESS$ denote the explained sum of squares (ESS) from regressing $Y$ ...
2
votes
0answers
64 views

Creating two clusters with as equal centroids as possible

I have a number of data points that can be placed in a 2D-space according to two variables, in this case named "Variable_1" and "Variable_2", like so: ...
4
votes
1answer
161 views

Clustering into teams of fixed size

There is a particular team-based video game that exposes a ladder of individual ratings for each player that looks like this (player, rating, wins, losses): A, 2000, 35, 12 B, 1900, 41, 19 C, 1800, ...
1
vote
0answers
23 views

Partitioning Spatial Dependence?

So, I have a group of nations that exhibit global spatial dependence. I want to partition the nations into groups so that, at an intuitive level at least, the "sum of the spatial dependence within ...
5
votes
4answers
3k views

How to split dataset for time-series prediction?

I have historic sales data from a bakery (daily, over 3 years). Now I want to build a model to predict future sales (using features like weekday, weather variables, etc.). How should I split the ...
0
votes
1answer
94 views

What is an equi-depth partition of the data?

In the paper Outlier Detection for High Dimensional Data at the beginning of section 1.3 Is written: Each attribute of the data is divided into $\phi$ equi-depth ranges. Thus, each range contains ...
1
vote
2answers
91 views

How can I partition a distribution into two sub-populations with fixed bias? (simulation)

I am trying to simulate a selection model for a variable $Y$ dependent on covariate vector $X$, so that two groups/sub-sets $S=(0,1)$ of observations on $Y$ are created, which have a fixed difference ...
4
votes
2answers
115 views

Assigning even partitions for Cross-Validation

This is a very basic question about cross-validation. Say that I have a sample size of 2901(or any difficult to divide number). How do I split this up into equal partitions (other than n=1)? And how ...
0
votes
1answer
97 views

Need help understanding response from Metis

I was wondering if any of you could help me understand the response I got from this clustering algorithm (Metis). As you probably can see, I'm trying to cluster IP addresses based on common records ...
2
votes
1answer
249 views

How do multi-attribute edge-weights influence community detection?

My graph consists of a computer network topology where each vertex is a physical node/device (depicted using its IP address). Two vertices will have an edge if the nodes have had communication with ...
0
votes
0answers
2k views

Using a Decision Tree Algorithm such as C4.5 to understand population Partition

I have multivariate data about a certain population with more than 1000 attributes per exemplar. Some of the variables are basic demographics attributes including: gender, age, race, ethnicity, ...
5
votes
1answer
628 views

Nested ANOVA: Unequal sample sizes? Variance components?

I am completely out of my depth on this, and all the reading I try to do just confuses me. I'm hoping you can explain things to me in a way that makes sense. (As always seems to be the case, "It ...
2
votes
0answers
221 views

Partitioning variance from logistic regression

Short version How can I partition the variance from the different levels in a nested mixed-effects logistic regression? Preferably using R, but even general principles would be helpful as a start. ...
1
vote
0answers
1k views

is it possible to use partitioned data(train&test) together with cross-fold validation?

I have used SPSS Clementine in order to train a classifier, for this I have used a partition node with 2 parts(train and test),then using a c5-tree and cross-fold validation. I did this because I ...
1
vote
0answers
1k views

R procedure for comparing multiple categorical variables (similar to anova() followed by t.test() for continuous)?

Big Picture: How can I implement partitioned Chi Square in R? I understand how to perform the overall Chi square, and then how to get individual parameters (observed counts, expected counts, ...
6
votes
2answers
2k views

Does Newman's network modularity work for signed, weighted graphs?

The modularity of a graph is defined on its Wikipedia page. In a different post, somebody explained that modularity can easily be computed (and maximized) for weighted networks because the adjacency ...
1
vote
2answers
438 views

Class labels in data partitions

Suppose that one partitions the data to training/validation/test sets for further application of some classification algorithm, and it happens that training set doesn't contain all class labels that ...
2
votes
1answer
251 views

How interactions between explanatory variables can be found using binary recursive partitioning?

I would like to investigate interactions between my explanatory variables prior to building a statsitical model. Apparently it is possible to do it in R using a regression tree (library: tree). This ...
1
vote
2answers
320 views

Modularity of graph: why are probabilities of self-loop included?

I'm trying to understand the Newman Modularity (doi:10.1073/pnas.0601602103) by investigating its calculation on the Wikipedia example . My question is why are probabilities of self-loops included in ...
2
votes
1answer
3k views

Correct way of evaluating Random Forest performance wrt training/test, feature selection, ntrees, random seed

I need to use Random Forest in my experiments. Although using the same training and test datasets, each time that I train the Random Forest on my training set, I get a different result on my test set. ...
4
votes
0answers
148 views

Variance partitioning - why be cautious?

I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...