Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

  7. current community

    your communities

    more stack exchange communities

    company blog
Cross Validated

Questions tagged [subset]

The tag has no usage guidance.

88 questions
info Newest Frequent Votes Active Unanswered
0
votes
1answer
15 views

Dividing datasets into fully-correlating subsets

I have a dataset of $N$ variables of which I know that (at least) one combination of their disjoint subsets results in non-negative correlation coefficients for all variables in these subsets. The ...
0
votes
0answers
17 views

Consistency metric explanation

I am trying to understand a bit more about the consistency metric (to understand how consistency-based subset evaluation works). I find on this paper the following equation : $$\text{Consistency}_s =...
0
votes
0answers
20 views

When should a regression be run across the subsets of a population vs the entire population

At work we are looking to run some models on subsets of a population. For the sake of this post, I'll call the models created on subsets 'conditional' and the model run on the entire population '...
0
votes
0answers
28 views

Finding p-value of similarity between 2 networks

So, I want to calculate p-value of a case if possible. Suppose I have: set A with 1,295 elements set B with 493 elements number of same element between set A and B (intersection) is 478 From each ...
1
vote
1answer
48 views

Are the assumptions of subset selection violated of all the data comes from a single person?

I’m interested in using machine learning techniques such as subset selection, lasso and ridge regression to predict which words an individual kid will get wrong. I have about 300 predictors and all my ...
1
vote
0answers
45 views

Probability that 3 out of 4 sets have non-empty intersection

I have $4$ randomly sampled subsets from population set $S = \{1,2,...,100\}$. Each subset has size $24$. What is the probability that at least one element is common among $3$ out of $4$ subsets?
3
votes
0answers
53 views

Dealing with many NA's in very large datasets for Lasso

I have a few very large and quite "dirty" (survey) datasets. Primarily, there are lots of NA's. These NA's are mostly the result of different questions being asked in different waves. It is perfectly ...
1
vote
1answer
27 views

finding expected value when a set is given and a subset of size n is chosen

I found an interesting coding challenge on pramp by a friend but I couldn't do it in time. Anyhow, it says given a set { 3,14,7,22,29,33} and random 3 element subset is generated each time and its ...
2
votes
0answers
104 views

Vapnik-Chervonenkis Dimension

My question has to do with the VC dimension of the class of convex polygons with $m$ vertices. A solution to this problem is given in the following: Advanced Algorithms. Call the class of all ...
0
votes
1answer
32 views

Why low error indicates better model when only one feature is used?

In this lecture, starting from first minute, there is an example of features for housing application. Let's say number of features is 1. I understand why lowest error idnicates better model, but why ...
1
vote
0answers
22 views

how to create a subset of models? [closed]

I want to select a specific models from a list of models generated by: m2 <- glm.nb(NroAnimales ~ anio + diaJuliano + I(diaJuliano^2) + BFmediana , data =datos, na.act ModelSel<-dredge(m2,rank=...
0
votes
0answers
23 views

Comparison with subsets of distribution

I am working with sequencing data and I have read density at codon/amino acid positions across different mRNAs. I am comparing the distribution of read densities for one amino acid with another. I use ...
0
votes
0answers
12 views

Using mean estimates of subsets of data or entire data to estimate a parameter

"Suppose that we wish to invest a fixed sum of money in two financial assets that yield returns of X and Y , respectively, where X and Y are random quantities. We will invest a fraction α of our money ...
1
vote
1answer
56 views

How to create a subset that respect the same distribution of the original data set?

I have a presumably easy question for you, but I am a real newbie in statistics, so please be patient. I have a data set containing N values that more or less follows a normal distribution. I need to ...
0
votes
0answers
41 views

hyperparameter optimization, subsets and congruency

This is a binary classification problem, GAMBoost model. The goal is to predict wether or not prices will rise after an event. The results are weighted and there is a measure of weighted hits, ...
0
votes
0answers
32 views

Hurst exponent on uneven data sets

I am currently implementing the hurst exponent for an experiment I am running (mostly for my own learning) using R. Following this fantastic explanation of the Hurst Exponent, I am having trouble ...
0
votes
3answers
69 views

Is this an example of a multiple comparisons problem?

I am conducting a comprehensive research determining the effect of 3 independent variables on the metabolic rate of an organism. However, the set-up of my data analysis leads me to believe that I ...
3
votes
0answers
75 views

How many subsets have n values greater/lesser than values in remaining subset?

For example, given set of ten numbers: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} Split into two equal-sized subsets. Multichoose calculates 252 configurations of subsets with 5 values (pick 5 from 10). Only ...
1
vote
0answers
78 views

Similarity metric for sets of sets of nominal data

I felt it important to include the full motivation for my question, since part of my trouble I think is simply not knowing the terms to search for. Similarity of set of sets and Similarity metric ...
2
votes
1answer
63 views

ML method to identify data subset with lower average values?

I have a task where I need to find anomalous groups of observations [y,X] by the average value of y, using the feature set X. A good interpretation is that these are customers and I want to find the ...
0
votes
0answers
29 views

Best subsets regression question regarding error? how can I proceed? (Minitab)

So I have 11 predictors and one response variable. I am wanting to look at the relationship (see what predictors effect the variable) and as I can't put all my predictors into multiple regression, I ...
1
vote
0answers
16 views

Subset with largest information gain

I am competing in a programming contest where the submission phase can be stated abstractly as follows : There is a known universe set, $U$, and a hidden target $T \subset U$. I submit $S \subset U$, ...
4
votes
4answers
307 views

Variance of set of subsets

First of all sorry for the sloppy terminology, but I am right looking for the name of a statistical concept. I was asked to calculate the "turnover" of the Facebook friends commenting on my posts, ...
0
votes
1answer
102 views

Decompose Covariance by Observations

Suppose I observe $n$ iid realizations of two random variables $X$ and $Y$, denoted respectively $x_i$ and $y_i$. Observations can be groupped into two subsamples, with $n_1$ and $n_2$ observations. I ...
0
votes
0answers
85 views

How to test whether subgroup is influential in overall group mean? (whether removing a subgroup affects results/all-but-subgroup mean != overall mean)

This question is similar to this one but I want to test whether the all-but-subgroup mean differs from overall group including the subgroup. This is DIFFERENT FROM THAT QUESTION in that the question ...
3
votes
0answers
191 views

How can I perform a two-sample multivariate t-test where one group is a subset of the other?

Is it possible to compare two means (ideally multivariate) where one comes from a group that is the subset of the other group? E.g. the first group consists of A, B, C, D, E subjects and the second ...
0
votes
0answers
18 views

A method to determine if a bivariate dataset contains two subsets

I have a set of data based on scores using the ECR Questionnaire. Each case has two scores, an aggregate Anxiety score, and an aggregate Avoidance score. I have centred the scores around their ...
1
vote
0answers
45 views

Heteroscedasticity in Data, none in Subset

This might seem like a trvial questions to some but sadly not me. My dataset has heteroscedasticity and therefore I use a heteroscedasticity consistent standard error estimator (I use the RLM macro ...
0
votes
1answer
93 views

Small values are sorted incorrectly, subset() gives wrong data in R [closed]

I have a (gwas-)data where one column is p-value. P-values vary from $1*10^{-8}$ to $1$. I would want to have a subset where I have only values where p-value is $5*10^{-5}$ or lower. I have the ...
1
vote
1answer
26 views

Comparing N-way joint frequency distributions of different subsets?

Say I have binary variables: tall basketball player football player vegan programmer student We are also given subsets of a population. My objective is that I want find out what combination of ...
1
vote
0answers
12 views

Store outputs of Lm performed on conditional dataframe subsets?

I am trying to store the regression's slopes of two variables recorded for different individuals through a season. What I would like is the slope of the regression for each individual for each day (...
0
votes
0answers
48 views

Bias in forward stepwise selection in regression

Let $Y=X\beta+E$ be a linear model with p variables and suppose that we perform som subset selection and we only use k variables where $k<p$. I want to get the biase os the new model. I call $X=(...
1
vote
0answers
114 views

Find a subset of input data space with high classification accuracy?

I'm having what would seem to be quite a general and widely applicable problem but I can't find literature on the subject. Example problem: Imagine I have medical records of thousands of patients (...
1
vote
0answers
9 views

How to properly select elements to tag manually for building training and test sets?

I have to select a subset (short texts like comments and twitter posts) out of a huge population to tag them manually for building a sentiment analysis training and test set. How should I select these ...
1
vote
0answers
402 views

ISLR book ridge/lasso regression cross-validation

I work with the book 'Introduction to statistical learning with R', link. I got a question about the cross-validation for ridge and lasso regressions (ellipsis indicate that part of the original code ...
2
votes
1answer
436 views

Subset Sum with Constraints

I'm taking a look at what I have identified as being a subset sum problem. I have around 650 potential values which could form part of my single sum value. I've began looking into the typical ...
2
votes
1answer
228 views

Using blocking in propensity score analysis?

I'm considering blocking my analysis dataset on one or two key variables prior to a standard sub-classification propensity score analysis. For example, if my dataset (which contains member level pre- ...
12
votes
2answers
169 views

Bias in average age for grandmaster title qualification by age groups?

It has been known for quite some time that the youngest age at which chess players managed to qualify for the grandmaster title has significantly decreased since the 1950s, and there are currently ...
0
votes
0answers
199 views

R: subsetting survey design fails after raking

all. I hope you can help me with a problem I encountered while analyzing data from a survey. I have actually done all the analysis, but after that obtained more reliable data about the population ...
1
vote
1answer
47 views

Unable to figure out equation with set probabilities

I am currently self-studying with Casella & Berger and there is a problem that has me stumped. Please note I am NOT asking for a solution, I just need clarification on the terms used. Here is the ...
1
vote
0answers
40 views

Significance for the whole dataset of detections in a subsample

I have a fairly simple question, but I am not so familiar with these problems so I would like to ask. If I have, let's say, 4 data samples belonging to the same overall population. If in a test a ...
0
votes
1answer
463 views

Predicting from RPart using a subset of variables

I fitted a RPart model from all of a couple dozen variables. Now, I want to test predictions using just a couple of variables that seem significant to me. If it was the iris dataset, for example, I ...
0
votes
1answer
194 views

Membership probability for distributions over subsets

Fix a set $\mathfrak{A}$ and a probability distribution $\mathcal{D} : 2^\mathfrak{A} \to [0,1]$ defined over subsets of $\mathfrak{A}$. We can then ask "what is the probability $p(x)$ that a subset ...
3
votes
2answers
123 views

Is it needed to train the selected model again on entire data before putting in production?

When a dataset is needed to be modeled, the process is to take a part of it out as holdout set which is "unseen" by the training method and is used to test the performance of models created using ...
0
votes
1answer
57 views

Testing variables in meta-analysis

I am conducting a meta-analysis of cognitive-behavioural interventions on self-esteem and I could use some assistance navigating through the statistical procedures. I have extracted pre-post scores ...
3
votes
1answer
650 views

How does one go about determining the standard deviation of an entire sample data set?

I have frequently been faced with analyzing csv files that are limited in length to approximately 32,000 records in length (overflows at 32,768, from signed integer limits on a 16 bit wide data bus). ...
0
votes
1answer
28 views

Subselect Package R - The covariance/total matrix supplied is not symmetric?

I was trying out the Subselect R package to see how it worked and if it would be useful for a logistic regression problem I'm working on. Link to the package. I decided I would follow Example 4 on ...
0
votes
1answer
2k views

Can forward and backward selection give the same model?

I'm using forward stepwise selection and backward stepwise selection to produce models in R. However, I got the exactly same model for two different methods. Is that normal? ...
0
votes
0answers
29 views

Subset of Pareto/powerlaw distribution

Given a set of random data generated using pareto-distribution, how can I get X% of this random data without losing pareto-distribution. In other words, how to select a subset of pareto-distribution ...
3
votes
1answer
113 views

Covariances of a random variable and its subset

Suppose event B is a subset of event A. $P(A) = p$ and $P(B) = q$. What is the Covariance of indicator functions from A and B. $$ \mathbb{1}_{A}(\omega) = \begin{cases} 1 & \omega \in A\\ 0 &...

1 2