Questions tagged [subset]
1 set, A, is a subset of another, B, iff all elements of A are elements of B.
101
questions
2
votes
2answers
192 views
Determine how similar groups of data are to each other
I have a pool of 15,000 unique* items. From that set, there are roughly 3200 sets of 60 items, selected non-randomly (in fact, for the sake of the framing of the question, let's say that each item is ...
0
votes
1answer
10 views
Inferential considerations when comparing a group against the total
I want to compare the value of a subgroup against the same value in the total population in a regression setting. The easiest way to do it would be to treat the subgroup and the total dataset as two ...
9
votes
3answers
946 views
How to calculate number of sets in Sigma Algebra
The example 1.2.2 of the book Statistical Inference by Casella and Berger states: if S has n elements, there are 2^n sets...(please see attached).
Could you please explain how the authors derived ...
0
votes
1answer
27 views
relation between conditional probability of subset and set
Suppose $A\subseteq B $. Can we derive a relationship between P(X|A) and P(X|B) for some X? X is associated with A.
0
votes
0answers
19 views
Data sub-set question
I have a data set of 15,000 with the following characteristics:
median : 3.3
avg : 4.3
st dev : 3.5
kurtosis : 19.05
skew : 3.94
Now from within that I have a smaller sub-set. There are 1,160 in ...
1
vote
0answers
39 views
Why do subset selection linear models have higher variance than the full model?
I do not get the meaning of this sentences from the Elements of Statistical Learning book when talking about subset selection methods before introducing shrinkage methods.
By retaining a subset of ...
1
vote
0answers
23 views
L sets and Probability theory
I am required to prove the following:
Let $L_1$ be the space of real-valued random variables on $(\Omega,\mathscr{A},\mathbb{P})$ which have finite expectation $\mathbb{E}(|X|)<\infty$ and let $...
0
votes
0answers
36 views
Problem with significance of results from regression on subset of data
I have performed a hedonic regression with a database where all the variables are very significant and according to the calculations made in R with a squared R = 0.6123.
I have performed the same ...
0
votes
0answers
22 views
Represent roles intersections
I have 11 roles, who defines rights (read, edit, write) on my application. For exemple I have "Inventory" who contains ressources_read and ...
0
votes
0answers
12 views
Converting extrapolation point to interpolation point
Just wondering if this is possible via mathematical conversion. I thought some Kernels might do this in the kernel space.
I got curious whether it's possible to create a general method that ...
0
votes
1answer
40 views
What statistical distribution test to use for comparing Dataset and its subset? [duplicate]
I have two sets, A and B ,containing height of students. Set A contains height of all students in the class and Set B contains height of some students from the same class. Hence, Set B is a subset of ...
0
votes
0answers
14 views
When shall I normalize/standarize the variables of a dataset? before or after subsetting a dataset?
I am using a subset of a large dataset to run panel regressions in R. Because variables range differently, I have to re-escale them (normalize/standarize). The problem is that I do not know whether I ...
0
votes
0answers
8 views
It is possible to calculate what the expected hazards ratio would be for a subgroup if I am assuming the subgroup is an independent variable?
I am interpreting a placebo-controlled comparative study. For the primary endpoint:
Overall
n=8179
Hazards ratio: 0.75, 95% CI: 0.68 - 0.83
Gender is assumed to be an independent variable in ...
0
votes
1answer
25 views
Which is the event, sample and population, notation confusion
I am really struggling with the notation for this. Below is the slide material followed by what I think it means/where I get stuck.
"Some unknown real world quantity $\Theta$ takes values in $\Omega$....
0
votes
0answers
31 views
Determining whether it matters which subset I pick from my dataset
I'm going to simplify my own research problem with the following example:
Lets suppose I have N=40 students in my classroom. I need to select 20 of them to participate in a national math competition. ...
0
votes
1answer
13 views
Resample dataframe by random subset of years
I have a dataset of 59 years worth of daily rainfall data.
I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
1
vote
0answers
26 views
Sample subsets with distribution proportional to elementwise product
I have a set of $n$ elements, $x_1,x_2,\dots,x_n$, each with a weight $w_1, w_2,\dots, w_n$. I want to sample a subset of fixed size $k$ such that the probability of drawing $\{x_a,\dots,x_b\}$ is ...
2
votes
1answer
137 views
Unsupervised clustering of sequence of events to subsequences
I have a big dataset of M sequences of [1 - N] events, where each event has multiple properties (start date, end date, location, ...
0
votes
1answer
38 views
Why is the relationship between max(X,Y) and X and Y the way it is?
My textbook says that the above follows from the observation:
$\{W\leq w\}=\{X\leq w\},\{Y\leq w\}$,
How do we prove the above observation?
0
votes
1answer
30 views
Survival analysis with incomplete data
I would like to estimate the association between exposure to an environmental contaminant and all-cause mortality. Unfortunately the dataset I can get access to is incomplete - exposure data are only ...
0
votes
0answers
22 views
A non random subset of a distribution is similar to the original distribution, why?
I have near 1 million correlations between two groups of variables. The distribution looks normal for the distributions of interest (centered around 0 and symmetric):
Using some other methods (not ...
1
vote
0answers
15 views
k-means and (re?) standardisation of a sub-set
I have data which is customer purchases of items in each of three months:
I have summed the data over the three months for each customer;
calculated the proportion of purchases that each item ...
0
votes
1answer
25 views
Dividing datasets into fully-correlating subsets
I have a dataset of $N$ variables of which I know that (at least) one combination of their disjoint subsets results in non-negative correlation coefficients for all variables in these subsets. The ...
0
votes
0answers
26 views
Consistency metric explanation
I am trying to understand a bit more about the consistency metric (to understand how consistency-based subset evaluation works).
I find on this paper the following equation :
$$\text{Consistency}_s =...
0
votes
0answers
34 views
When should a regression be run across the subsets of a population vs the entire population
At work we are looking to run some models on subsets of a population. For the sake of this post, I'll call the models created on subsets 'conditional' and the model run on the entire population '...
0
votes
0answers
67 views
Finding p-value of similarity between 2 networks
So, I want to calculate p-value of a case if possible.
Suppose I have:
set A with 1,295 elements
set B with 493 elements
number of same element between set A and B (intersection) is 478
From each ...
1
vote
1answer
57 views
Are the assumptions of subset selection violated of all the data comes from a single person?
I’m interested in using machine learning techniques such as subset selection, lasso and ridge regression to predict which words an individual kid will get wrong. I have about 300 predictors and all my ...
1
vote
0answers
49 views
Probability that 3 out of 4 sets have non-empty intersection
I have $4$ randomly sampled subsets from population set $S = \{1,2,...,100\}$. Each subset has size $24$.
What is the probability that at least one element is common among $3$ out of $4$ subsets?
3
votes
0answers
76 views
Dealing with many NA's in very large datasets for Lasso
I have a few very large and quite "dirty" (survey) datasets. Primarily, there are lots of NA's. These NA's are mostly the result of different questions being asked in different waves. It is perfectly ...
1
vote
1answer
124 views
finding expected value when a set is given and a subset of size n is chosen
I found an interesting coding challenge on pramp by a friend but I couldn't do it in time. Anyhow, it says given a set { 3,14,7,22,29,33} and random 3 element subset is generated each time and its ...
2
votes
0answers
124 views
Vapnik-Chervonenkis Dimension
My question has to do with the VC dimension of the class of convex polygons with $m$ vertices. A solution to this problem is given in the following: Advanced Algorithms.
Call the class of all ...
0
votes
1answer
45 views
Why low error indicates better model when only one feature is used?
In this lecture, starting from first minute, there is an example of features for housing application. Let's say number of features is 1. I understand why lowest error indicates better model, but why ...
1
vote
1answer
223 views
How to create a subset that respect the same distribution of the original data set?
I have a presumably easy question for you, but I am a real newbie in statistics, so please be patient. I have a data set containing $N$ values that more or less follows a normal distribution. I need ...
0
votes
3answers
100 views
Is this an example of a multiple comparisons problem?
I am conducting a comprehensive research determining the effect of 3 independent variables on the metabolic rate of an organism. However, the set-up of my data analysis leads me to believe that I ...
3
votes
0answers
76 views
How many subsets have n values greater/lesser than values in remaining subset?
For example, given set of ten numbers:
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Split into two equal-sized subsets. Multichoose calculates 252 configurations of subsets with 5 values (pick 5 from 10).
Only ...
2
votes
0answers
167 views
Similarity metric for sets of sets of nominal data
I felt it important to include the full motivation for my question, since part of my trouble I think is simply not knowing the terms to search for.
Similarity of set of sets
and
Similarity metric ...
2
votes
1answer
64 views
ML method to identify data subset with lower average values?
I have a task where I need to find anomalous groups of observations [y,X] by the average value of y, using the feature set X.
A good interpretation is that these are customers and I want to find the ...
1
vote
0answers
17 views
Subset with largest information gain
I am competing in a programming contest where the submission phase can be stated abstractly as follows : There is a known universe set, $U$, and a hidden target $T \subset U$. I submit $S \subset U$, ...
4
votes
4answers
391 views
Variance of set of subsets
First of all sorry for the sloppy terminology, but I am right looking for the name of a statistical concept.
I was asked to calculate the "turnover" of the Facebook friends commenting on my posts, ...
0
votes
1answer
122 views
Decompose Covariance by Observations
Suppose I observe $n$ iid realizations of two random variables $X$ and $Y$, denoted respectively $x_i$ and $y_i$. Observations can be groupped into two subsamples, with $n_1$ and $n_2$ observations. I ...
0
votes
0answers
120 views
How to test whether subgroup is influential in overall group mean? (whether removing a subgroup affects results/all-but-subgroup mean != overall mean)
This question is similar to this one but I want to test whether the all-but-subgroup mean differs from overall group including the subgroup. This is DIFFERENT FROM THAT QUESTION in that the question ...
2
votes
0answers
248 views
How can I perform a two-sample multivariate t-test where one group is a subset of the other?
Is it possible to compare two means (ideally multivariate) where one comes from a group that is the subset of the other group?
E.g. the first group consists of A, B, C, D, E subjects and the second ...
1
vote
0answers
46 views
Heteroscedasticity in Data, none in Subset
This might seem like a trvial questions to some but sadly not me.
My dataset has heteroscedasticity and therefore I use a heteroscedasticity consistent standard error estimator (I use the RLM macro ...
0
votes
1answer
102 views
Small values are sorted incorrectly, subset() gives wrong data in R [closed]
I have a (gwas-)data where one column is p-value. P-values vary from $1*10^{-8}$ to $1$.
I would want to have a subset where I have only values where p-value is $5*10^{-5}$ or lower.
I have the ...
1
vote
1answer
26 views
Comparing N-way joint frequency distributions of different subsets?
Say I have binary variables:
tall
basketball player
football player
vegan
programmer
student
We are also given subsets of a population.
My objective is that I want find out what combination of ...
1
vote
0answers
15 views
Store outputs of Lm performed on conditional dataframe subsets?
I am trying to store the regression's slopes of two variables recorded for different individuals through a season.
What I would like is the slope of the regression for each individual for each day (...
0
votes
0answers
59 views
Bias in forward stepwise selection in regression
Let $Y=X\beta+E$ be a linear model with p variables and suppose that we perform som subset selection and we only use k variables where $k<p$.
I want to get the biase os the new model.
I call $X=(...
1
vote
0answers
146 views
Find a subset of input data space with high classification accuracy?
I'm having what would seem to be quite a general and widely applicable problem but I can't find literature on the subject.
Example problem: Imagine I have medical records of thousands of patients (...
1
vote
0answers
11 views
How to properly select elements to tag manually for building training and test sets?
I have to select a subset (short texts like comments and twitter posts) out of a huge population to tag them manually for building a sentiment analysis training and test set. How should I select these ...
1
vote
0answers
530 views
ISLR book ridge/lasso regression cross-validation
I work with the book 'Introduction to statistical learning with R', link. I got a question about the cross-validation for ridge and lasso regressions (ellipsis indicate that part of the original code ...
