Questions tagged [subset]

One set, A, is a subset of another, B, if and only if all elements of A are elements of B.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

How to filter in R with subsets [migrated]

I have a dataframe with columns: coast, weight gain, length gain, release weight, release length. Coast is categorical ("E" or "W") and the other variables are numeric. I would ...
7
votes
1answer
94 views

Is there a name for the increase in variance upon remeasurement after subsetting with a cut-off value?

Context: My problem relates to estimating effect sizes, such as Cohen's d, when looking at a subset of the population defined by a cut-off threshold. This effect size is the difference in two ...
2
votes
1answer
25 views

When is subsetting survey data prior to analysis NOT a problem?

Survey researchers are typically advised to not subset their data prior to analysis because it will produce incorrect variance estimates. My understanding of the inferential issue is that removing ...
0
votes
0answers
17 views

Need help understanding paper by Jolliffe - various methods in choosing subsets for PCA

There's a paper I'm trying to base some work off of called Variable Selection and the Interpretation of Principal Subspaces which uses some various methods to choose subsets of variables. Their ...
0
votes
0answers
7 views

Uncertainty when taking the difference between a measurement and a subset measurement

Lets say I collected $\mathbf{N_t}$ data points for an experiment and this resulted in a measurement of $A\pm\sigma_A$. Then I'm interested in how a particular variable effects the study and remove $\...
0
votes
0answers
14 views

Maximizing a subset's variance

Consider the given set [x_1, ... , x_n]. For a fixed natural number M, I want a subset [x_(i1), ..., x_(iM)], whose cardinality ...
1
vote
0answers
78 views

Why do subset selection linear models have higher variance than the full model?

I do not get the meaning of this sentences from the Elements of Statistical Learning book when talking about subset selection methods before introducing shrinkage methods. By retaining a subset of ...
0
votes
0answers
14 views

Compare models from different data subsets [duplicate]

I'm looking to compare linear regression models from different data subsets in r. The models are not nested. I have a model from the complete dataset with sex as a factor and then 2 separate models ...
3
votes
2answers
67 views

Is it preferable to subset data to test specific hypotheses or specify a full model and run contrasts?

Let's say I have a 2x2 design where participants are either in condition A or condition B and, within each condition, either get exposed to exposure C or exposure D. First, I want to test whether ...
0
votes
1answer
831 views

Predicting from RPart using a subset of variables

I fitted a RPart model from all of a couple dozen variables. Now, I want to test predictions using just a couple of variables that seem significant to me. If it was the iris dataset, for example, I ...
0
votes
1answer
33 views

Logistic regression balanced dataset

I am quite inexperienced using logistic regression and am having trouble understanding my data and how the regression behaves. Here's the outline of my problem: I have a (medical) test that gives a ...
1
vote
0answers
32 views

How to treat apples and oranges best in multiple regression analysis? One fits all regression model or separate sub models? [duplicate]

I have a data set ready for multiple regression analysis that consist of apples and oranges. Let's say the depended variable is fruit size and there is a bunch of independent variables (categorical &...
2
votes
2answers
25 views

Relationship between deterministc function of random variables

Given a discrete $P(X,Y,Z)$ let's call $\Omega$ the set of all deterministic functions $f: XYZ \rightarrow W$ and $\Omega'$ the set of all deterministic functions $f': XY \rightarrow V$. Is it correct ...
0
votes
0answers
12 views

How to test if the subset is representative to whole dataset

I would like to ask how to test if the subset is representative to the whole dataset. I have for example 150 values in whole "population" and I selected (not randomly) 30 of them. I would ...
4
votes
0answers
110 views

Dealing with many NA's in very large datasets for Lasso

I have a few very large and quite "dirty" (survey) datasets. Primarily, there are lots of NA's. These NA's are mostly the result of different questions being asked in different waves. It is ...
0
votes
0answers
16 views

Reduced chi-squared for models fit to different subsets of data

I have some data to which I am fitting piecewise linear models. I want to select different subsets of the data, fit a model to each of them, and then compare which subsets are best able to be ...
0
votes
0answers
9 views

Subset dataframe by conditioning on two observations within a column for each subject (group)

I am looking to compare the results of a survey (not shown) before and after teaching. You can see below that the last column column specifies whether this observation contains the information from &...
2
votes
0answers
352 views

How can I perform a two-sample multivariate t-test where one group is a subset of the other?

Is it possible to compare two means (ideally multivariate) where one comes from a group that is the subset of the other group? E.g. the first group consists of A, B, C, D, E subjects and the second ...
0
votes
0answers
11 views

Check if a subset is significantly different from the dataset

I have a dataset with violence rate per City that can be grouped by State. I need to check if any State has significantly different violence rates than the entire dataset. How could I achieve that? ...
0
votes
1answer
18 views

How to isolate a linearly correlating subset from data?

I have a data set of a few thousand data points, and some of them show strong linear correlation. I would like to isolate these data points into their own subsets. What would be the best approach for ...
0
votes
0answers
73 views

GLM in R: Subset Covariate to specific factor Levels

I like to conduct a GLM in R like this: glm ( Y ~ X + Z + a + b + c + d, quasipoisson(link="log")) with Y as dependent Variable (Count Data, integer) X as ...
0
votes
0answers
27 views

Difference between stratification and subset

This may be a duplicate question asked differently, I am curious to know the difference between performing an analysis by stratification on a variable and by subsetting the dataset based on that ...
4
votes
1answer
390 views

Probability problem - Probability to pick a subset divisible by 3

I'm trying to solve this puzzle but I get stuck. I thought about trying to use the law of total probability to solve intermediate problems with subset of size k but it didn't helped me that much. Is ...
1
vote
1answer
143 views

Decompose Covariance by Observations

Suppose I observe $n$ iid realizations of two random variables $X$ and $Y$, denoted respectively $x_i$ and $y_i$. Observations can be groupped into two subsamples, with $n_1$ and $n_2$ observations. I ...
4
votes
1answer
249 views

Unsupervised clustering of sequence of events to subsequences

I have a big dataset of M sequences of [1 - N] events, where each event has multiple properties (start date, end date, location, ...
1
vote
1answer
24 views

Unsupervised Subspace learning for different Tasks

once we extract subspaces(subset of features) for a dataset using unsupervised learning, can we use them for any task I mean for classification, clustering, or outlier detection? Is the selection of ...
1
vote
0answers
105 views

Q: AUC of each subgroup is smaller than overall AUC

I have a validation data set of 29242 patients, with known labels/health outcomes and predictions that were generated by some model. 28626 patients are negative and 616 are positive The overall AUC is ...
3
votes
1answer
471 views

R: subsetting survey design fails after raking

all. I hope you can help me with a problem I encountered while analyzing data from a survey. I have actually done all the analysis, but after that obtained more reliable data about the population ...
1
vote
1answer
70 views

What is it called when we “zoom in” a data distribution?

Suppose we have a 2D distribution of shops in an area. For example, here is a heatmap density plot of (latitude, longitude) location of shops in a province: Now suppose that we "zoom into" the ...
3
votes
0answers
66 views

Is it appropriate to include overlapping sub-scale scores and total-scale scores in a meta-analysis?

Question, brief In a single multi-level meta-analysis, is it appropriate to include subscale scores (which represent part of a measure) in addition to total scale scores (representing the full ...
0
votes
1answer
69 views

How to test that a subset of data is distributed no differently than original (discrete, not normal)

In the research I am doing we have initially had ~600 samples which had a certain distribution (not normal, although close). The characteristic of these samples that I am interested in is discrete (...
0
votes
1answer
35 views

Compact encoding (vectorization) of unbounded sets

Question I have a set of sets. Each set is unbounded. I would like to find a methodology to encode (vectorize) each subset. I am more specifically interested in memory efficient solutions. ...
2
votes
1answer
681 views

ISLR book ridge/lasso regression cross-validation

I work with the book 'Introduction to statistical learning with R', link. I got a question about the cross-validation for ridge and lasso regressions (ellipsis indicate that part of the original code ...
2
votes
2answers
607 views

Determine how similar groups of data are to each other

I have a pool of 15,000 unique* items. From that set, there are roughly 3200 sets of 60 items, selected non-randomly (in fact, for the sake of the framing of the question, let's say that each item is ...
1
vote
0answers
42 views

Sample subsets with distribution proportional to elementwise product

I have a set of $n$ elements, $x_1,x_2,\dots,x_n$, each with a weight $w_1, w_2,\dots, w_n$. I want to sample a subset of fixed size $k$ such that the probability of drawing $\{x_a,\dots,x_b\}$ is ...
1
vote
1answer
11 views

Inferential considerations when comparing a group against the total

I want to compare the value of a subgroup against the same value in the total population in a regression setting. The easiest way to do it would be to treat the subgroup and the total dataset as two ...
9
votes
3answers
1k views

How to calculate number of sets in Sigma Algebra

The example 1.2.2 of the book Statistical Inference by Casella and Berger states: if S has n elements, there are 2^n sets...(please see attached). Could you please explain how the authors derived ...
4
votes
1answer
37k views

Applying linear regression to a data subset [closed]

I've just run a linear regression on an entire data set, but now I need to run the regression with data just from females within the data. Females are denoted under the ...
0
votes
1answer
45 views

relation between conditional probability of subset and set

Suppose $A\subseteq B $. Can we derive a relationship between P(X|A) and P(X|B) for some X? X is associated with A.
1
vote
0answers
24 views

L sets and Probability theory

I am required to prove the following: Let $L_1$ be the space of real-valued random variables on $(\Omega,\mathscr{A},\mathbb{P})$ which have finite expectation $\mathbb{E}(|X|)<\infty$ and let $...
0
votes
0answers
64 views

Problem with significance of results from regression on subset of data

I have performed a hedonic regression with a database where all the variables are very significant and according to the calculations made in R with a squared R = 0.6123. I have performed the same ...
0
votes
0answers
27 views

Represent roles intersections

I have 11 roles, who defines rights (read, edit, write) on my application. For exemple I have "Inventory" who contains ressources_read and ...
0
votes
1answer
289 views

What statistical distribution test to use for comparing Dataset and its subset? [duplicate]

I have two sets, A and B ,containing height of students. Set A contains height of all students in the class and Set B contains height of some students from the same class. Hence, Set B is a subset of ...
0
votes
1answer
28 views

Which is the event, sample and population, notation confusion

I am really struggling with the notation for this. Below is the slide material followed by what I think it means/where I get stuck. "Some unknown real world quantity $\Theta$ takes values in $\Omega$....
0
votes
0answers
36 views

Determining whether it matters which subset I pick from my dataset

I'm going to simplify my own research problem with the following example: Lets suppose I have N=40 students in my classroom. I need to select 20 of them to participate in a national math competition. ...
3
votes
2answers
3k views

Proper variable selection: Use only training data or full data?

I'm going through the lab exercises in "Introduction to Statistical Learning" and am having difficulty understanding the proper way to do best subset selection. On page 248, it states that: ... We ...
6
votes
1answer
140 views

Foundational sufficient statistics

I've been reading through Casella and Berger's Statistical Infererence and have am having a little trouble understanding something in their explanation of sufficient statistics. Here is the passage ...
0
votes
1answer
16 views

Resample dataframe by random subset of years

I have a dataset of 59 years worth of daily rainfall data. I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
0
votes
1answer
46 views

Why low error indicates better model when only one feature is used?

In this lecture, starting from first minute, there is an example of features for housing application. Let's say number of features is 1. I understand why lowest error indicates better model, but why ...
0
votes
1answer
42 views

Why is the relationship between max(X,Y) and X and Y the way it is?

My textbook says that the above follows from the observation: $\{W\leq w\}=\{X\leq w\},\{Y\leq w\}$, How do we prove the above observation?