The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
3answers
29 views

How to split dataset for time-series prediction?

I have historic sales data from a bakery (daily, over 3 years). Now I want to build a model to predict future sales (using features like weekday, weather variables, etc.). How should I split the ...
0
votes
0answers
9 views

One-hot encoding before or after split into training and test sets?

Is it safe to do my one-hot encoding of categorical variables before splitting the data into training and test and cross validation sets? I ask because this process alters the dimension of the data, ...
0
votes
0answers
9 views

What is an equi-depth partition of the data?

In the paper Outlier Detection for High Dimensional Data at the beginning of section 1.3 Is written: Each attribute of the data is divided into $\phi$ equi-depth ranges. Thus, each range contains ...
1
vote
2answers
71 views

How can I partition a distribution into two sub-populations with fixed bias? (simulation)

I am trying to simulate a selection model for a variable $Y$ dependent on covariate vector $X$, so that two groups/sub-sets $S=(0,1)$ of observations on $Y$ are created, which have a fixed difference ...
3
votes
2answers
71 views

Assigning even partitions for Cross-Validation

This is a very basic question about cross-validation. Say that I have a sample size of 2901(or any difficult to divide number). How do I split this up into equal partitions (other than n=1)? And how ...
0
votes
1answer
19 views

Need help understanding response from Metis

I was wondering if any of you could help me understand the response I got from this clustering algorithm (Metis). As you probably can see, I'm trying to cluster IP addresses based on common records ...
1
vote
1answer
51 views

How do multi-attribute edge-weights influence community detection?

My graph consists of a computer network topology where each vertex is a physical node/device (depicted using its IP address). Two vertices will have an edge if the nodes have had communication with ...
0
votes
0answers
352 views

Using a Decision Tree Algorithm such as C4.5 to understand population Partition

I have multivariate data about a certain population with more than 1000 attributes per exemplar. Some of the variables are basic demographics attributes including: gender, age, race, ethnicity, ...
4
votes
0answers
110 views

Nested ANOVA: Unequal sample sizes? Variance components?

I am completely out of my depth on this, and all the reading I try to do just confuses me. I'm hoping you can explain things to me in a way that makes sense. (As always seems to be the case, "It ...
1
vote
0answers
44 views

Partitioning variance from logistic regression

Short version How can I partition the variance from the different levels in a nested mixed-effects logistic regression? Preferably using R, but even general principles would be helpful as a start. ...
1
vote
0answers
349 views

is it possible to use partitioned data(train&test) together with cross-fold validation?

I have used SPSS Clementine in order to train a classifier, for this I have used a partition node with 2 parts(train and test),then using a c5-tree and cross-fold validation. I did this because I ...
1
vote
0answers
251 views

R procedure for comparing multiple categorical variables (similar to anova() followed by t.test() for continuous)?

Big Picture: How can I implement partitioned Chi Square in R? I understand how to perform the overall Chi square, and then how to get individual parameters (observed counts, expected counts, ...
0
votes
0answers
24 views

Neural Net on results from Partition Bootstrap Forest - validity?

I am seeing some promising results with this method for my 'nutrition' dataset but am wary about modelling on results of a model. Problenms with this data I cannot seem to reduce my many correlated ...
2
votes
2answers
288 views

Does Newman's network modularity work for signed, weighted graphs?

The modularity of a graph is defined on its Wikipedia page. In a different post, somebody explained that modularity can easily be computed (and maximized) for weighted networks because the adjacency ...
0
votes
0answers
27 views

Investigating 'minor' effect variables?

Is there any way to investigate minor contributing $X$ variables in a model when there are one or two $X$ variables which contribute to the explanation of a majority of the variation in the $Y$ ...
1
vote
2answers
90 views

Class labels in data partitions

Suppose that one partitions the data to training/validation/test sets for further application of some classification algorithm, and it happens that training set doesn't contain all class labels that ...
2
votes
1answer
106 views

How interactions between explanatory variables can be found using binary recursive partitioning?

I would like to investigate interactions between my explanatory variables prior to building a statsitical model. Apparently it is possible to do it in R using a regression tree (library: tree). This ...
1
vote
1answer
111 views

Modularity of graph: why are probabilities of self-loop included?

I'm trying to understand the Newman Modularity (doi:10.1073/pnas.0601602103) by investigating its calculation on the Wikipedia example . My question is why are probabilities of self-loops included in ...
3
votes
0answers
104 views

Variance partitioning - why be cautious?

I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...
1
vote
1answer
375 views

Partitioning Around Medoids

I have a question regarding Partitioning Around Medoids (PAM) clustering algorithm, because everywhere I look, it is described differently. In every step of the algorithms do I swap only one medoid or ...
0
votes
0answers
27 views

Testing that a finite quantity is equally partitioned

Suppose I have a set of observations on a finite (continuously divisible) quantity $Q$ (say $Q = 100$ to be concrete). For each observation $i$, $Q$ is partitioned into three parts $(A_i, B_i, C_i)$, ...
0
votes
0answers
58 views

Discretization of skewed data (time durations)

I have data that describes the duration of how long a person views a webpage. This is quite varied and in the context wherein I gathered the data, it was very skewed. People mostly spent short amounts ...
1
vote
1answer
113 views

Multivariate Gaussian with 3 Partitions

Given that we partition a Gaussian random vector $\textbf{x}$ into three groups, $\textbf{x}_a$, $\textbf{x}_b$, and $\textbf{x}_c$, with a corresponding partitioning of the mean vector $\mu$ and of ...
0
votes
2answers
116 views

Linear SVM C optimisation data, how to partition into train, model construction, test

My aim is to find the best C for a linear SVM classification using libsvm, I have 120 instances in total and 2 classes which I want to classify. I have question regarding partitioning a the dataset ...
3
votes
1answer
226 views

Is $R^2$ value valid for insignificant OLS regression model?

I am interested in stating that ___ % of the variance in Y is explained uniquely by $X_1$ and ___ % is explained uniquely by $X_2$. Is there some way to obtain this from a multiple regression ...
1
vote
0answers
83 views

How to Partition An Interaction SS in ANOVA

I'm a new user of R and I'm trying to replicate Table 6.18 on page 262 in Statistical Procedures in Agricultural Research, By K. A. Gomez and A. A. Gomez. New York, Chichester, etc.: Wiley (1984). ...
2
votes
1answer
171 views

Can I partition a frequency table and use Fisher's exact test instead of the chi-squared test?

I am working with very small cell counts, and am wondering if Fisher's exact test can be used in place of the chi-squared test when partitioning IxJ tables. I am interested if the same process for ...
2
votes
1answer
349 views

mobForest R Package

I have recently begĂșn to learn about model based recursive partitioning by playing around with MOB in the party package. I came across this mobForest package but am a little baffled towards what it is ...
2
votes
2answers
231 views

Statistical independence of combining random samples

Given a single finite population of size N, if I obtain S random samples from the data (replacing after each one), and randomly partition these into 2 sets of S/2 random samples, can I treat these two ...
2
votes
1answer
227 views

Bootstrap confidence intervals for partitioned variances in R

I'm trying to determine the variance partitioning within plant drought tolerance data among hierarchical ecological levels, from species to forest sites to biome levels. I did that with the varcomp ...
4
votes
2answers
1k views

How do I find similarities between two sets of data

I have a group of data with 12 different football players, and they are rated for 11 different skills (speed, skill, flair, etc). I am looking to pair up individuals based on similar footballers, and ...
0
votes
1answer
262 views

Is there a statistically good way to split a set of data into (generally) uneven groups?

So here's the scenario: I have about 100 or so "items", each assigned with a number and total guesses... this number is a guess of what people think the item is worth (in terms of money). This number ...
4
votes
1answer
85 views

Can complex sampling be ignored for exploratory patient-centered analyses?

Hello fellow StackExchange users, Preamble: I have been tasked with performing a cluster analysis (or possibly a latent class analysis, as I am pondering) to find non-overlapping groups of like ...
5
votes
2answers
236 views

Number of events of type B given n events of type A in a Poisson process

I have a simple probability calculation I was working on that I came up with an answer for, but a question a colleague asked me led me to come up with a second approach - and a different answer. I put ...
4
votes
2answers
190 views

Problem of “clustering” into most similar groups

I do not think that the following problem can be solved with k-means clustering. I am not sure though. Okay, let me describe the problem. I need to find a way or an algorithm that groups members of a ...
0
votes
1answer
93 views

Where can I find the articles discussing about the suggested training-data partitioning ratio?

Right now, I'm doing my thesis which use multilayer perceptron learning method to train a model. What I learned from my class is the purpose to partition the data which can be separated into 3 group ...
1
vote
1answer
1k views

How to use the hier.part package for hierarchical partitioning?

Hello after struggling with using R for the last couple of days I was hoping someone could help me with a statistical analysis I am completing for an environmental science honours project. Using R ...
4
votes
1answer
276 views

Estimate the population variance from a set of means

I have a set of measurements which is partitioned into M partitions. However, I only have the partition sizes $N_i$ and the means $\bar{x}_i$ from each partition. Because all measurements are assumed ...
9
votes
2answers
2k views

party vs. rpart vs. ??? for partitioning trees in R

It's been a while since I looked at partitioning trees. Last time I did this sort of thing, I like party in R (created by Hothorn). The idea of conditional inference via sampling makes sense to me. ...
2
votes
1answer
208 views

Stationary distribution on a partition of the state space?

I would like to use the following type of model and am wondering if someone could let me know if this technique has been studied before (I assume it has) and where to learn more about it? The idea is ...
2
votes
0answers
171 views

Establishing a dynamic threshold for partitioning data

Given a sorted set of data points, where each point represents the length of a common substring between two files, I want to systematically decide a cut off point where I can start considering them to ...
4
votes
1answer
739 views

Difference in implementation of binary splits in decision trees

I am curious about the practical implementation of a binary split in a decision tree - as it relates to levels of a categorical predictor $X{j}$. Specifically, I often will utilize some sort of ...
2
votes
1answer
117 views

calculate the rate of change

I posted this on maths, but seems it would be better here :S http://math.stackexchange.com/questions/49941/calculate-the-rate-of-change Basically I am trying to calculate the change frequency for ...
6
votes
1answer
1k views

Interpreting output of igraph's fastgreedy.community clustering method

With the help of several people in this community I have been wetting my feet in clustering some social network data using igraph's implementation of modularity-based clustering. I am having some ...
22
votes
7answers
8k views

How to do community detection in a weighted social network/graph?

I'm wondering if someone could suggest what are good starting points when it comes to performing community detection/graph partitioning/clustering on a graph that has weighted, undirected edges. The ...
7
votes
2answers
2k views

Newman's modularity clustering for graphs

I am interested in running Newman's modularity clustering algorithm on a large graph. If you can point me to a library (or R package, etc) that implements it I would be most grateful. best ~lara