Questions tagged [subset]
One set, A, is a subset of another, B, if and only if all elements of A are elements of B.
120
questions
0
votes
0
answers
21
views
Model comparison on different datasets + Non-nested model comparison
I actually have two questions. Similar questions unfortunatelly do not answer mine.
1)I realize that with anova(model1,model2) one can only compare two models if ...
0
votes
0
answers
14
views
comparing data subset to whole
I am trying to write a story about my local high school and its gender diversity in AP programs. I have ethnic data for everyone enrolled, and for everyone in AP programs at the school. What/how is ...
0
votes
0
answers
27
views
Should I use AIC / BIC or rather cross validation for discovering gov. equations through linear regression (SINDy)?
I want to use linear regression with very large design matrix for discovery of governing equations to i.e. physical systems. The design matrix would include potential terms that can be part of the ...
- 11
0
votes
0
answers
8
views
Variance inequality for nested sets
Let $X, Y,$ and $Z$ are three random variables/vectors, and let $f(., ., .)$ is a real-valued, deterministic function. If $Z$ is independent of $\{X, Y\}$ (e.g., $X, Y, Z$ are independent) then
\begin{...
- 789
1
vote
1
answer
46
views
Linear Regression - Data Subset with Lowest Mean-Squared Error?
Short version: given a linear regression dataset and an integer $K$, what data subset of size $K$ results in linear parameters with the lowest mean squared error on the entire dataset?
Long version: ...
- 699
0
votes
1
answer
18
views
Determine the likelihood of occurrence of an activity within a subset
I have a monitor that measures dust concentrations around our neighborhood, and from this dataset I subset it into three scenarios to see which scenarios create the most dust. However, when I do this, ...
- 1
0
votes
0
answers
29
views
Test subset data with extra variable against predicted value from top model built using full dataset
I am trying to test if a response variable is predicted by spatial/temporal variables + genetics, but I only have genetics data for 25% of the data. Can I test if genetics play a role in the following ...
- 101
0
votes
0
answers
24
views
Need help understanding paper by Jolliffe - various methods in choosing subsets for PCA
There's a paper I'm trying to base some work off of called Variable Selection and the Interpretation of Principal Subspaces which uses some various methods to choose subsets of variables. Their ...
- 1
2
votes
1
answer
43
views
When is subsetting survey data prior to analysis NOT a problem?
Survey researchers are typically advised to not subset their data prior to analysis because it will produce incorrect variance estimates. My understanding of the inferential issue is that removing ...
- 359
0
votes
0
answers
10
views
Uncertainty when taking the difference between a measurement and a subset measurement
Lets say I collected $\mathbf{N_t}$ data points for an experiment and this resulted in a measurement of $A\pm\sigma_A$. Then I'm interested in how a particular variable effects the study and remove $\...
- 1
0
votes
0
answers
19
views
Maximizing a subset's variance
Consider the given set [x_1, ... , x_n].
For a fixed natural number M, I want a subset [x_(i1), ..., x_(iM)], whose cardinality ...
URYSOHN
0
votes
0
answers
17
views
Compare models from different data subsets [duplicate]
I'm looking to compare linear regression models from different data subsets in r. The models are not nested. I have a model from the complete dataset with sex as a factor and then 2 separate models ...
- 31
3
votes
2
answers
197
views
Is it preferable to subset data to test specific hypotheses or specify a full model and run contrasts?
Let's say I have a 2x2 design where participants are either in condition A or condition B and, within each condition, either get exposed to exposure C or exposure D.
First, I want to test whether ...
- 770
0
votes
1
answer
45
views
Logistic regression balanced dataset
I am quite inexperienced using logistic regression and am having trouble understanding my data and how the regression behaves. Here's the outline of my problem:
I have a (medical) test that gives a ...
- 3
7
votes
1
answer
108
views
Is there a name for the increase in variance upon remeasurement after subsetting with a cut-off value?
Context: My problem relates to estimating effect sizes, such as Cohen's d, when looking at a subset of the population defined by a cut-off threshold. This effect size is the difference in two ...
- 1,866
1
vote
0
answers
34
views
How to treat apples and oranges best in multiple regression analysis? One fits all regression model or separate sub models? [duplicate]
I have a data set ready for multiple regression analysis that consist of apples and oranges. Let's say the depended variable is fruit size and there is a bunch of independent variables (categorical &...
- 473
3
votes
2
answers
32
views
Relationship between deterministc function of random variables
Given a discrete $P(X,Y,Z)$ let's call $\Omega$ the set of all deterministic functions $f: XYZ \rightarrow W$ and $\Omega'$ the set of all deterministic functions $f': XY \rightarrow V$. Is it correct ...
- 617
0
votes
1
answer
26
views
How to isolate a linearly correlating subset from data?
I have a data set of a few thousand data points, and some of them show strong linear correlation. I would like to isolate these data points into their own subsets. What would be the best approach for ...
- 11
0
votes
0
answers
57
views
Difference between stratification and subset
This may be a duplicate question asked differently, I am curious to know the difference between performing an analysis by stratification on a variable and by subsetting the dataset based on that ...
4
votes
1
answer
911
views
Probability problem - Probability to pick a subset divisible by 3
I'm trying to solve this puzzle but I get stuck. I thought about trying to use the law of total probability to solve intermediate problems with subset of size k but it didn't helped me that much. Is ...
- 49
1
vote
1
answer
24
views
Unsupervised Subspace learning for different Tasks
once we extract subspaces(subset of features) for a dataset using unsupervised learning, can we use them for any task I mean for classification, clustering, or outlier detection?
Is the selection of ...
- 309
1
vote
1
answer
121
views
What is it called when we "zoom in" a data distribution?
Suppose we have a 2D distribution of shops in an area. For example, here is a heatmap density plot of (latitude, longitude) location of shops in a province:
Now suppose that we "zoom into" the ...
- 308
3
votes
0
answers
113
views
Is it appropriate to include overlapping sub-scale scores and total-scale scores in a meta-analysis?
Question, brief
In a single multi-level meta-analysis, is it appropriate to include subscale scores (which represent part of a measure) in addition to total scale scores (representing the full ...
- 303
1
vote
0
answers
211
views
Q: AUC of each subgroup is smaller than overall AUC
I have a validation data set of 29242 patients, with known labels/health outcomes and predictions that were generated by some model. 28626 patients are negative and 616 are positive
The overall AUC is ...
- 39
0
votes
1
answer
202
views
How to test that a subset of data is distributed no differently than original (discrete, not normal)
In the research I am doing we have initially had ~600 samples which had a certain distribution (not normal, although close). The characteristic of these samples that I am interested in is discrete (...
- 1
0
votes
1
answer
43
views
Compact encoding (vectorization) of unbounded sets
Question
I have a set of sets. Each set is unbounded.
I would like to find a methodology to encode (vectorize) each subset.
I am more specifically interested in memory efficient solutions.
...
- 127
2
votes
2
answers
1k
views
Determine how similar groups of data are to each other
I have a pool of 15,000 unique* items. From that set, there are roughly 3200 sets of 60 items, selected non-randomly (in fact, for the sake of the framing of the question, let's say that each item is ...
- 85
1
vote
1
answer
11
views
Inferential considerations when comparing a group against the total
I want to compare the value of a subgroup against the same value in the total population in a regression setting. The easiest way to do it would be to treat the subgroup and the total dataset as two ...
- 2,313
10
votes
3
answers
2k
views
How to calculate number of sets in Sigma Algebra
The example 1.2.2 of the book Statistical Inference by Casella and Berger states: if S has n elements, there are 2^n sets...(please see attached).
Could you please explain how the authors derived ...
- 317
0
votes
1
answer
48
views
relation between conditional probability of subset and set
Suppose $A\subseteq B $. Can we derive a relationship between P(X|A) and P(X|B) for some X? X is associated with A.
- 103
2
votes
1
answer
145
views
Why do subset selection linear models have higher variance than the full model?
I do not get the meaning of this sentences from the Elements of Statistical Learning book when talking about subset selection methods before introducing shrinkage methods.
By retaining a subset of ...
- 551
1
vote
0
answers
25
views
L sets and Probability theory
I am required to prove the following:
Let $L_1$ be the space of real-valued random variables on $(\Omega,\mathscr{A},\mathbb{P})$ which have finite expectation $\mathbb{E}(|X|)<\infty$ and let $...
- 11
0
votes
0
answers
109
views
Problem with significance of results from regression on subset of data
I have performed a hedonic regression with a database where all the variables are very significant and according to the calculations made in R with a squared R = 0.6123.
I have performed the same ...
0
votes
0
answers
27
views
Represent roles intersections
I have 11 roles, who defines rights (read, edit, write) on my application. For exemple I have "Inventory" who contains ressources_read and ...
- 1
0
votes
1
answer
632
views
What statistical distribution test to use for comparing Dataset and its subset? [duplicate]
I have two sets, A and B ,containing height of students. Set A contains height of all students in the class and Set B contains height of some students from the same class. Hence, Set B is a subset of ...
0
votes
1
answer
28
views
Which is the event, sample and population, notation confusion
I am really struggling with the notation for this. Below is the slide material followed by what I think it means/where I get stuck.
"Some unknown real world quantity $\Theta$ takes values in $\Omega$....
- 113
0
votes
0
answers
37
views
Determining whether it matters which subset I pick from my dataset
I'm going to simplify my own research problem with the following example:
Lets suppose I have N=40 students in my classroom. I need to select 20 of them to participate in a national math competition. ...
- 101
0
votes
1
answer
21
views
Resample dataframe by random subset of years
I have a dataset of 59 years worth of daily rainfall data.
I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
1
vote
0
answers
51
views
Sample subsets with distribution proportional to elementwise product
I have a set of $n$ elements, $x_1,x_2,\dots,x_n$, each with a weight $w_1, w_2,\dots, w_n$. I want to sample a subset of fixed size $k$ such that the probability of drawing $\{x_a,\dots,x_b\}$ is ...
- 11
4
votes
1
answer
334
views
Unsupervised clustering of sequence of events to subsequences
I have a big dataset of M sequences of [1 - N] events, where each event has multiple properties (start date, end date, location, ...
- 318
0
votes
1
answer
46
views
Why is the relationship between max(X,Y) and X and Y the way it is?
My textbook says that the above follows from the observation:
$\{W\leq w\}=\{X\leq w\},\{Y\leq w\}$,
How do we prove the above observation?
- 122
0
votes
1
answer
42
views
Survival analysis with incomplete data
I would like to estimate the association between exposure to an environmental contaminant and all-cause mortality. Unfortunately the dataset I can get access to is incomplete - exposure data are only ...
- 315
1
vote
0
answers
18
views
k-means and (re?) standardisation of a sub-set
I have data which is customer purchases of items in each of three months:
I have summed the data over the three months for each customer;
calculated the proportion of purchases that each item ...
- 165
0
votes
1
answer
37
views
Dividing datasets into fully-correlating subsets
I have a dataset of $N$ variables of which I know that (at least) one combination of their disjoint subsets results in non-negative correlation coefficients for all variables in these subsets. The ...
- 1
0
votes
0
answers
124
views
Finding p-value of similarity between 2 networks
So, I want to calculate p-value of a case if possible.
Suppose I have:
set A with 1,295 elements
set B with 493 elements
number of same element between set A and B (intersection) is 478
From each ...
- 143
1
vote
1
answer
101
views
Are the assumptions of subset selection violated of all the data comes from a single person?
I’m interested in using machine learning techniques such as subset selection, lasso and ridge regression to predict which words an individual kid will get wrong. I have about 300 predictors and all my ...
- 67
1
vote
0
answers
61
views
Probability that 3 out of 4 sets have non-empty intersection
I have $4$ randomly sampled subsets from population set $S = \{1,2,...,100\}$. Each subset has size $24$.
What is the probability that at least one element is common among $3$ out of $4$ subsets?
- 121
4
votes
0
answers
137
views
Dealing with many NA's in very large datasets for Lasso
I have a few very large and quite "dirty" (survey) datasets. Primarily, there are lots of NA's. These NA's are mostly the result of different questions being asked in different waves. It is ...
- 239
1
vote
1
answer
595
views
finding expected value when a set is given and a subset of size n is chosen
I found an interesting coding challenge on pramp by a friend but I couldn't do it in time. Anyhow, it says given a set { 3,14,7,22,29,33} and random 3 element subset is generated each time and its ...
- 11
2
votes
0
answers
172
views
Vapnik-Chervonenkis Dimension
My question has to do with the VC dimension of the class of convex polygons with $m$ vertices. A solution to this problem is given in the following: Advanced Algorithms.
Call the class of all ...
