Questions tagged [aggregation]
Refers to "lumping together" potentially inhomogeneous groups of data.
255
questions
0
votes
0
answers
15
views
Total generalized variance for Box-Cox transformed components
I have a couple Gaussian mixture models where each component comes from (component-wise) Box-Cox transformed data. These models do not describe the same data: the individual components are selected ...
- 11
1
vote
1
answer
42
views
Monthly data with seasonality: is linear regression appropriate?
I have the following situation: there is data for six years, per month. In a year you can see variation per month, since the data is influenced by the seasons. Also, there is an upward trend over the ...
- 11
0
votes
0
answers
11
views
Aggregating predictions on micro data
I am a machine-learning noob, so please bear with me.
I am trying to predict the aggregate number of businesses that will exit (i.e. shut down permanently) in the next quarter (or year). However, my ...
0
votes
0
answers
11
views
Aggregating metrics at parent
I have a model with several independent variables/metrics that each additionally has a corresponding unit, pre-defined upper/lower bounds and a desired value.
Now I wish to compute some basic ...
0
votes
0
answers
9
views
Aggregate predictions to generalize on large scale
I actually work on medical datas to predict outcome after treatment on patients with metastasis lesions of carcinomas.
Each patient have different number of lesions with some with like 30 lesions and ...
- 3
1
vote
0
answers
27
views
How can aggregation be helpful in mitigating bias?
I am working on the estimation assessing the impact of exposure to infrastructure (mainly schooling) on the number of children.
Since I do not have migration data, my colleague recommended that I ...
- 61
0
votes
1
answer
25
views
Wondering if averaging datapoints by a set number of observations is a valid way to aggregate data in a scatterplot
So essentially the problem I have is this: I've created an economics simulation which, among other things, tracks the profit rates and capital intensity of all the firms over the course of the ...
- 1
0
votes
1
answer
42
views
Aggregation of daily correlations to different frequencies
After using the DCC-GARCH model to find the estimated correlations between 2 time series, I would like to perform linear regression with these daily correlations as my dependent variable and test the ...
0
votes
0
answers
59
views
Multiple regression with aggregate data
I have a set of panel data and want to do a fixed-effect regression. My data consists of four years of survey data with (nearly) identical structure.
My dependent variable (for each year) is a ...
- 1
0
votes
0
answers
20
views
Conducting macro-level OLS regression using aggregated data with few level 1 observations
I am currently conducting a study where the outcome variable is measured only at the level 2/group level. The dependent variable is a measure of the proportion of all municipal spending that is being ...
0
votes
0
answers
25
views
Best statistical model for ranked preferences
I need some help figuring out the best statistical model for my planned research.
Here is the setup: I want to understand support for four distinct policies. More specifically, I want to understand ...
- 111
0
votes
0
answers
28
views
Correct degrees-of-freedom for a (semi)partial-correlation analysis that aggregates multi-level data with unknown dependencies?
I'm not sure what question to ask, so I'll start by explaining what I'm trying to do.
The data
I have a hierarchical dataset that contains multiple batches (5); Data points for each batch have three ...
- 251
2
votes
0
answers
27
views
Approaches to (non-linear) meta-regression to estimate the conditional population distribution of $y$ based on aggregate data
I want to estimate the distribution of a variable in certain subgroups of the population based on pooling of aggregate data reported in various observational studies. For simplicity, assume there is a ...
- 176
0
votes
0
answers
29
views
Predicting pair-level response with individual-level predictor
I want to fit a regression model with a pair-level measurement (i.e. litter size), but have an individual-level predictor variable (i.e. a morphometric measure = morph).
Basically, all variables are ...
- 23
0
votes
0
answers
47
views
Random tree partition of ordered lists
In a standard random tree, the input is a dataset of $N$ observations $x_1, x_2, \dots, x_N$ and the algorithm partitions the data in such a way that observations with similar values are grouped ...
- 11
0
votes
0
answers
28
views
How to address Simpson's paradox when aggregating ratios?
I have a table of means and ns from several studies comparing x and y groups. I've also computed the ratio of the means for each ...
- 190
1
vote
0
answers
41
views
Ordinal logistic regression with aggregated data
I'm wondering if it is feasible to conduct a ordinal logistic regression on the following dataset.
I want to investigate if there is a correlation between the alcohol outlet density (as counts) and ...
- 11
1
vote
0
answers
22
views
Best way to analyse two experiments
I'm looking for help/indication on approaching a data integration problem.
I have a dose-response curve that is described by a log curve. Because of measurement difficulties, the dose and response can'...
- 11
0
votes
0
answers
47
views
How to aggregate standard deviations of subsets robustly?
Assume I know the sample standard deviations $s_i, i=1..N$ of $N$ subsets of data.
I can assume that the subsets have the same size (usually $S=2$) and the same underlying real standard deviation $\...
- 115
0
votes
0
answers
48
views
Comparing means on individual observation level data or aggregated daily level data for two periods
I have data on purchases on individual customer level. The customers are segmented into different groups, like region. I want to compare sales from different periods.
Should I use the daily aggregated ...
- 1
1
vote
0
answers
47
views
Is kruskall wallis appropriate here?
I have collected campaign finance data about donations individuals have made to 3 groups (A, B, C) between 2015 and 2020. Each donor has only donated to one group but some donors have donated ...
- 11
2
votes
1
answer
98
views
Aggregated time series or regression approach?
The setup I describe below is analogous to my actual problem.
Problem: I have millions of individuals in my dataset and for each individual, I have certain stats over time. I.e. for one individual Joe,...
- 75
0
votes
0
answers
92
views
Loss of variance in aggregated data?
I have a nested dataset where information on individual workplace characteristics is available on the case level, and data on recorded sick leave on the group level.
7k individuals are nested within ...
- 1
1
vote
0
answers
108
views
Can I use logistic regression from individual-level survey data to make predictions at an aggregate level?
I have a logistic regression model that predicts the likelihood of an individual getting PTSD after a flood based on their race and gender. The regression is based on a survey of individual flood ...
- 11
0
votes
1
answer
293
views
weighted average in collaborative filtering
I studied on collaborative filtering recently and found that for ratings at last most methods applied were weighted average, no matter what they had proposed, similarity, time, etc.
Thus I am ...
- 11
0
votes
1
answer
34
views
Calculating average for each individual or not?
I have a question about how I should organize my data before doing some statistical analyses.
In my data, I have a sample of rats for which the level of tear stains and body lesions is measured. They ...
- 129
1
vote
2
answers
797
views
How to solve the problem of having sparse data that would become too small when aggregated?
I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
1
vote
1
answer
41
views
Aggregating and averaging time series data to have same nrow per group
I suspect the solution will be pretty straightforward but I'm struggling to figure it out.
The problem:
I have an experimental design with several video stimuli of different time lengths (between 90s ...
- 101
0
votes
0
answers
16
views
Handling time series data with same start and end datetime but different vector lengths
I am dealing with time series data with 15 different features from a machine. The thing is that the different feature datasets that have been collected have been done at different frequecies. That is ...
1
vote
0
answers
28
views
How does aggregation across time bring about feedback?
On page 11 Pankratz says: "Sometimes there is sound theoretical reasoning in favor of a one-way relationship from the inputs to the output (no feedback). But even then it is important to have a ...
- 4,176
0
votes
0
answers
142
views
Intuitive explanation of Choquet Integral for data aggregation
I've recently stumbled upon this python package that implements Choquet integral as a way of aggregating data. Does anyone have an intuitive way of explaining how does this integral work in this case?
- 1
0
votes
1
answer
113
views
Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two
I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
1
vote
0
answers
25
views
Staggered Average and Standard Deviation of incomplete dataset
I have an AxBxC matrix of measurements, from which an arbitrarily-sized-and-distributed number of measurements have been deemed invalid and removed (i.e. that matrix contains a mostly random ...
- 9
4
votes
1
answer
54
views
What type of statistical inferences are possible on aggregated tables?
I have a table with the following format
CountH SubcountH1 SubcountH2 CountK...
VarA
SubA1
SubA2
SubA3
VarB
SubB1
SubB2
.
.
.
The table is filled with ...
- 402
2
votes
0
answers
211
views
How to calculate aggregate standard deviation for composite scores in a meta-analysis
I wonder if someone would be kind and able to provide guidance/suggestions on a stats issue that myself and colleagues are trying to handle while performing a systematic review and meta-analysis.
In ...
0
votes
0
answers
54
views
Aggregating forecasts and their standard errors over time periods
Good day all,
I have an ARIMA model which gives me a weekly forecast and standard error for each period.
If I were to aggregate the forecasts into monthly time-windows (for simplicity sake, let's ...
- 101
1
vote
0
answers
29
views
Combined multiple within-group effect sizes
I'm conducting a meta-analysis on observational studies examined the association of age with different drug groups prescription and in order to combine the effect sizes of all drug groups together, a ...
- 41
1
vote
1
answer
210
views
Is there a right way to aggregate timeseries data for anomaly detection and forecasting?
Hello Cross Validated Users,
I have had a question on my mind for a long time whenever I work with timeseries but I have never quite found the right answer.
Here is the thing: when creating an ...
- 111
0
votes
1
answer
74
views
How can I verify the correlation between appetizers and main meals
I have a data set of users. that looks something like this.
user_id
appetizer
main_meal
185
30
120
142
8
56
34
0
200
etc.
etc.
..
there are other columns but I don't believe it's relevant for my ...
- 123
0
votes
0
answers
44
views
Implications of using the mean of the max as a predictor in regression
What are the implications of using a predictor which has been aggregated twice (the mean of a set of max's) as a variable in a regression analysis?
I'm performing a regression on the panel dataset:
<...
- 45
0
votes
0
answers
54
views
Obtaining better estimates when you know that a set of input variables are independently and identically distributed
Suppose I have a family of random variables
$$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$
and I know how to sample $SomeDistribution_i$ for any $i$.
Suppose I also define a random variable $Y = f(...
- 409
0
votes
1
answer
34
views
How to assess performance when it greatly varies from year to year?
We have an annual performance metric for a number of educational institutions. The figure for any one institution varies a lot from year to year, as it depends very greatly on the student cohort, but ...
1
vote
0
answers
72
views
How to convert high to low frequency data
I have data for a macroeconomic variable, and it has monthly frequency but I need to work with it in quarters. Is there a statistical method to perform this frequency conversion that ensures the least ...
- 141
1
vote
1
answer
24
views
Does autocorrelation at granular intervals still exist at aggregated intervals?
I have a theoretical question autocorrelation at different sized intervals. Suppose I have a dataset at minutely intervals, that follows an AR(2) process and then drops into white noise at lags > 2....
- 151
0
votes
1
answer
29
views
Composite Indicator - Negative Correlation Issue
I'm trying to build a composite indicator that aims to measure poverty. I'm planning to aggregate a series of variables using PCA. However I have a doubt, all the variables I want to include are ...
- 13
1
vote
0
answers
69
views
Training Individual-Level Predictor using Distribution of Group-level Data
I have a problem in which I'm looking to train an individual-level predictor for outcomes. I have information on individual-level covariates, but I don't have individual-level outcome variables. ...
- 11
2
votes
1
answer
945
views
Level of Aggregation for Event Study (Staggered DiD) Analysis
This might be a bit basic, but I'm struggling to determine the ideal level of aggregation of the data for an Event Study analysis.
I wish to study if a specific policy affected fertility in certain ...
- 21
2
votes
3
answers
350
views
Preferred way to sum different time series together (in software)
Is there a canonical/best approach to computationally summing different time series together?
What I mean by that is the operation
$$
\sum_i{s_i(t)} = S(t)
$$
where $s_i(t)$ is the $i$-th time series, ...
2
votes
0
answers
134
views
What is the "appropriate" way to aggregate different proportions?
I've found a lot of ways -- arithmetic mean, geometric mean, harmonic mean -- and I've invented things which have no name (and, likely, deserved none). At the end I just get a number and it's hard to ...
- 345
3
votes
1
answer
320
views
Calculating an aggregated score
I'm out of my element when dealing with statistics, so I hope you'll be able to offer me some guidance.
I'm working on a project where students will apply for scholarships, and then a panel of people (...
- 131