Questions tagged [subset]

One set, A, is a subset of another, B, if and only if all elements of A are elements of B.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Reduced chi-squared for models fit to different subsets of data

I have some data to which I am fitting piecewise linear models. I want to select different subsets of the data, fit a model to each of them, and then compare which subsets are best able to be ...
0
votes
0answers
8 views

Subset dataframe by conditioning on two observations within a column for each subject (group)

I am looking to compare the results of a survey (not shown) before and after teaching. You can see below that the last column column specifies whether this observation contains the information from &...
0
votes
0answers
9 views

Check if a subset is significantly different from the dataset

I have a dataset with violence rate per City that can be grouped by State. I need to check if any State has significantly different violence rates than the entire dataset. How could I achieve that? ...
0
votes
1answer
15 views

How to isolate a linearly correlating subset from data?

I have a data set of a few thousand data points, and some of them show strong linear correlation. I would like to isolate these data points into their own subsets. What would be the best approach for ...
0
votes
0answers
30 views

GLM in R: Subset Covariate to specific factor Levels

I like to conduct a GLM in R like this: glm ( Y ~ X + Z + a + b + c + d, quasipoisson(link="log")) with Y as dependent Variable (Count Data, integer) X as ...
0
votes
0answers
14 views

Difference between stratification and subset

This may be a duplicate question asked differently, I am curious to know the difference between performing an analysis by stratification on a variable and by subsetting the dataset based on that ...
4
votes
1answer
221 views

Probability problem - Probability to pick a subset divisible by 3

I'm trying to solve this puzzle but I get stuck. I thought about trying to use the law of total probability to solve intermediate problems with subset of size k but it didn't helped me that much. Is ...
1
vote
1answer
24 views

Unsupervised Subspace learning for different Tasks

once we extract subspaces(subset of features) for a dataset using unsupervised learning, can we use them for any task I mean for classification, clustering, or outlier detection? Is the selection of ...
0
votes
0answers
8 views

Multi-categorical set balancing

I have a particular mathematical problem that I would name as multi-categorical set balancing. I don't think this is a new problem but I do not know the correct term for it, therefore I am also ...
1
vote
1answer
55 views

What is it called when we “zoom in” a data distribution?

Suppose we have a 2D distribution of shops in an area. For example, here is a heatmap density plot of (latitude, longitude) location of shops in a province: Now suppose that we "zoom into" the ...
3
votes
0answers
27 views

Is it appropriate to include overlapping sub-scale scores and total-scale scores in a meta-analysis?

Question, brief In a single multi-level meta-analysis, is it appropriate to include subscale scores (which represent part of a measure) in addition to total scale scores (representing the full ...
0
votes
0answers
55 views

Subset selection with reinforcement learning

I have a set of $N$ items of which a subset of arbitrary size can be chosen. I want a reinforcement learning (RL) agent to perform the subset selection and am unsure how to best design the action ...
1
vote
0answers
55 views

Q: AUC of each subgroup is smaller than overall AUC

I have a validation data set of 29242 patients, with known labels/health outcomes and predictions that were generated by some model. 28626 patients are negative and 616 are positive The overall AUC is ...
0
votes
1answer
37 views

How to test that a subset of data is distributed no differently than original (discrete, not normal)

In the research I am doing we have initially had ~600 samples which had a certain distribution (not normal, although close). The characteristic of these samples that I am interested in is discrete (...
0
votes
0answers
9 views

How can I redistribute the probability of a partially matching name based on frequency of unmatched word tokens?

I am a computer scientist, not a statistician, so I apologise in advance if I am unaware of common knowledge in this area or if I have explained something poorly. Please feel free to ask for ...
0
votes
1answer
34 views

Compact encoding (vectorization) of unbounded sets

Question I have a set of sets. Each set is unbounded. I would like to find a methodology to encode (vectorize) each subset. I am more specifically interested in memory efficient solutions. ...
0
votes
0answers
15 views

Combining PCAs group of variables

So I want to Research the importance of 6 Group of variables on my dependant variable. I have ~50 variables in 6 Groups (which i Need to keep for my hypothesis). I did a PCA on each Group of Variables ...
2
votes
2answers
311 views

Determine how similar groups of data are to each other

I have a pool of 15,000 unique* items. From that set, there are roughly 3200 sets of 60 items, selected non-randomly (in fact, for the sake of the framing of the question, let's say that each item is ...
1
vote
1answer
11 views

Inferential considerations when comparing a group against the total

I want to compare the value of a subgroup against the same value in the total population in a regression setting. The easiest way to do it would be to treat the subgroup and the total dataset as two ...
9
votes
3answers
1k views

How to calculate number of sets in Sigma Algebra

The example 1.2.2 of the book Statistical Inference by Casella and Berger states: if S has n elements, there are 2^n sets...(please see attached). Could you please explain how the authors derived ...
0
votes
1answer
44 views

relation between conditional probability of subset and set

Suppose $A\subseteq B $. Can we derive a relationship between P(X|A) and P(X|B) for some X? X is associated with A.
1
vote
0answers
55 views

Why do subset selection linear models have higher variance than the full model?

I do not get the meaning of this sentences from the Elements of Statistical Learning book when talking about subset selection methods before introducing shrinkage methods. By retaining a subset of ...
1
vote
0answers
24 views

L sets and Probability theory

I am required to prove the following: Let $L_1$ be the space of real-valued random variables on $(\Omega,\mathscr{A},\mathbb{P})$ which have finite expectation $\mathbb{E}(|X|)<\infty$ and let $...
0
votes
0answers
50 views

Problem with significance of results from regression on subset of data

I have performed a hedonic regression with a database where all the variables are very significant and according to the calculations made in R with a squared R = 0.6123. I have performed the same ...
0
votes
0answers
25 views

Represent roles intersections

I have 11 roles, who defines rights (read, edit, write) on my application. For exemple I have "Inventory" who contains ressources_read and ...
0
votes
1answer
175 views

What statistical distribution test to use for comparing Dataset and its subset? [duplicate]

I have two sets, A and B ,containing height of students. Set A contains height of all students in the class and Set B contains height of some students from the same class. Hence, Set B is a subset of ...
0
votes
1answer
25 views

Which is the event, sample and population, notation confusion

I am really struggling with the notation for this. Below is the slide material followed by what I think it means/where I get stuck. "Some unknown real world quantity $\Theta$ takes values in $\Omega$....
0
votes
0answers
36 views

Determining whether it matters which subset I pick from my dataset

I'm going to simplify my own research problem with the following example: Lets suppose I have N=40 students in my classroom. I need to select 20 of them to participate in a national math competition. ...
0
votes
1answer
15 views

Resample dataframe by random subset of years

I have a dataset of 59 years worth of daily rainfall data. I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
1
vote
0answers
38 views

Sample subsets with distribution proportional to elementwise product

I have a set of $n$ elements, $x_1,x_2,\dots,x_n$, each with a weight $w_1, w_2,\dots, w_n$. I want to sample a subset of fixed size $k$ such that the probability of drawing $\{x_a,\dots,x_b\}$ is ...
2
votes
1answer
205 views

Unsupervised clustering of sequence of events to subsequences

I have a big dataset of M sequences of [1 - N] events, where each event has multiple properties (start date, end date, location, ...
0
votes
1answer
41 views

Why is the relationship between max(X,Y) and X and Y the way it is?

My textbook says that the above follows from the observation: $\{W\leq w\}=\{X\leq w\},\{Y\leq w\}$, How do we prove the above observation?
0
votes
1answer
31 views

Survival analysis with incomplete data

I would like to estimate the association between exposure to an environmental contaminant and all-cause mortality. Unfortunately the dataset I can get access to is incomplete - exposure data are only ...
1
vote
0answers
17 views

k-means and (re?) standardisation of a sub-set

I have data which is customer purchases of items in each of three months: I have summed the data over the three months for each customer; calculated the proportion of purchases that each item ...
0
votes
1answer
34 views

Dividing datasets into fully-correlating subsets

I have a dataset of $N$ variables of which I know that (at least) one combination of their disjoint subsets results in non-negative correlation coefficients for all variables in these subsets. The ...
0
votes
0answers
89 views

Finding p-value of similarity between 2 networks

So, I want to calculate p-value of a case if possible. Suppose I have: set A with 1,295 elements set B with 493 elements number of same element between set A and B (intersection) is 478 From each ...
1
vote
1answer
78 views

Are the assumptions of subset selection violated of all the data comes from a single person?

I’m interested in using machine learning techniques such as subset selection, lasso and ridge regression to predict which words an individual kid will get wrong. I have about 300 predictors and all my ...
1
vote
0answers
53 views

Probability that 3 out of 4 sets have non-empty intersection

I have $4$ randomly sampled subsets from population set $S = \{1,2,...,100\}$. Each subset has size $24$. What is the probability that at least one element is common among $3$ out of $4$ subsets?
4
votes
0answers
95 views

Dealing with many NA's in very large datasets for Lasso

I have a few very large and quite "dirty" (survey) datasets. Primarily, there are lots of NA's. These NA's are mostly the result of different questions being asked in different waves. It is ...
1
vote
1answer
365 views

finding expected value when a set is given and a subset of size n is chosen

I found an interesting coding challenge on pramp by a friend but I couldn't do it in time. Anyhow, it says given a set { 3,14,7,22,29,33} and random 3 element subset is generated each time and its ...
2
votes
0answers
140 views

Vapnik-Chervonenkis Dimension

My question has to do with the VC dimension of the class of convex polygons with $m$ vertices. A solution to this problem is given in the following: Advanced Algorithms. Call the class of all ...
0
votes
1answer
46 views

Why low error indicates better model when only one feature is used?

In this lecture, starting from first minute, there is an example of features for housing application. Let's say number of features is 1. I understand why lowest error indicates better model, but why ...
1
vote
1answer
427 views

How to create a subset that respect the same distribution of the original data set?

I have a presumably easy question for you, but I am a real newbie in statistics, so please be patient. I have a data set containing $N$ values that more or less follows a normal distribution. I need ...
0
votes
3answers
124 views

Is this an example of a multiple comparisons problem?

I am conducting a comprehensive research determining the effect of 3 independent variables on the metabolic rate of an organism. However, the set-up of my data analysis leads me to believe that I ...
3
votes
0answers
78 views

How many subsets have n values greater/lesser than values in remaining subset?

For example, given set of ten numbers: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} Split into two equal-sized subsets. Multichoose calculates 252 configurations of subsets with 5 values (pick 5 from 10). Only ...
2
votes
0answers
233 views

Similarity metric for sets of sets of nominal data

I felt it important to include the full motivation for my question, since part of my trouble I think is simply not knowing the terms to search for. Similarity of set of sets and Similarity metric ...
2
votes
1answer
69 views

ML method to identify data subset with lower average values?

I have a task where I need to find anomalous groups of observations [y,X] by the average value of y, using the feature set X. A good interpretation is that these are customers and I want to find the ...
1
vote
0answers
19 views

Subset with largest information gain

I am competing in a programming contest where the submission phase can be stated abstractly as follows : There is a known universe set, $U$, and a hidden target $T \subset U$. I submit $S \subset U$, ...
5
votes
4answers
518 views

Variance of set of subsets

First of all sorry for the sloppy terminology, but I am right looking for the name of a statistical concept. I was asked to calculate the "turnover" of the Facebook friends commenting on my posts, ...
1
vote
1answer
136 views

Decompose Covariance by Observations

Suppose I observe $n$ iid realizations of two random variables $X$ and $Y$, denoted respectively $x_i$ and $y_i$. Observations can be groupped into two subsamples, with $n_1$ and $n_2$ observations. I ...