Questions tagged [aggregation]

Refers to "lumping together" potentially inhomogeneous groups of data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Total generalized variance for Box-Cox transformed components

I have a couple Gaussian mixture models where each component comes from (component-wise) Box-Cox transformed data. These models do not describe the same data: the individual components are selected ...
ladislaw94's user avatar
1 vote
1 answer
42 views

Monthly data with seasonality: is linear regression appropriate?

I have the following situation: there is data for six years, per month. In a year you can see variation per month, since the data is influenced by the seasons. Also, there is an upward trend over the ...
Eva's user avatar
  • 11
0 votes
0 answers
11 views

Aggregating predictions on micro data

I am a machine-learning noob, so please bear with me. I am trying to predict the aggregate number of businesses that will exit (i.e. shut down permanently) in the next quarter (or year). However, my ...
Mishal Ahmed's user avatar
0 votes
0 answers
11 views

Aggregating metrics at parent

I have a model with several independent variables/metrics that each additionally has a corresponding unit, pre-defined upper/lower bounds and a desired value. Now I wish to compute some basic ...
Track9250's user avatar
0 votes
0 answers
9 views

Aggregate predictions to generalize on large scale

I actually work on medical datas to predict outcome after treatment on patients with metastasis lesions of carcinomas. Each patient have different number of lesions with some with like 30 lesions and ...
Nicolas's user avatar
1 vote
0 answers
27 views

How can aggregation be helpful in mitigating bias?

I am working on the estimation assessing the impact of exposure to infrastructure (mainly schooling) on the number of children. Since I do not have migration data, my colleague recommended that I ...
Yendao Su's user avatar
0 votes
1 answer
25 views

Wondering if averaging datapoints by a set number of observations is a valid way to aggregate data in a scatterplot

So essentially the problem I have is this: I've created an economics simulation which, among other things, tracks the profit rates and capital intensity of all the firms over the course of the ...
Nicosims's user avatar
0 votes
1 answer
42 views

Aggregation of daily correlations to different frequencies

After using the DCC-GARCH model to find the estimated correlations between 2 time series, I would like to perform linear regression with these daily correlations as my dependent variable and test the ...
Jan-Willem van Boven's user avatar
0 votes
0 answers
59 views

Multiple regression with aggregate data

I have a set of panel data and want to do a fixed-effect regression. My data consists of four years of survey data with (nearly) identical structure. My dependent variable (for each year) is a ...
Jan's user avatar
  • 1
0 votes
0 answers
20 views

Conducting macro-level OLS regression using aggregated data with few level 1 observations

I am currently conducting a study where the outcome variable is measured only at the level 2/group level. The dependent variable is a measure of the proportion of all municipal spending that is being ...
user386406's user avatar
0 votes
0 answers
25 views

Best statistical model for ranked preferences

I need some help figuring out the best statistical model for my planned research. Here is the setup: I want to understand support for four distinct policies. More specifically, I want to understand ...
YouLocalRUser's user avatar
0 votes
0 answers
28 views

Correct degrees-of-freedom for a (semi)partial-correlation analysis that aggregates multi-level data with unknown dependencies?

I'm not sure what question to ask, so I'll start by explaining what I'm trying to do. The data I have a hierarchical dataset that contains multiple batches (5); Data points for each batch have three ...
MRule's user avatar
  • 251
2 votes
0 answers
27 views

Approaches to (non-linear) meta-regression to estimate the conditional population distribution of $y$ based on aggregate data

I want to estimate the distribution of a variable in certain subgroups of the population based on pooling of aggregate data reported in various observational studies. For simplicity, assume there is a ...
user9794's user avatar
  • 176
0 votes
0 answers
29 views

Predicting pair-level response with individual-level predictor

I want to fit a regression model with a pair-level measurement (i.e. litter size), but have an individual-level predictor variable (i.e. a morphometric measure = morph). Basically, all variables are ...
BRB's user avatar
  • 23
0 votes
0 answers
47 views

Random tree partition of ordered lists

In a standard random tree, the input is a dataset of $N$ observations $x_1, x_2, \dots, x_N$ and the algorithm partitions the data in such a way that observations with similar values are grouped ...
pengzell's user avatar
0 votes
0 answers
28 views

How to address Simpson's paradox when aggregating ratios?

I have a table of means and ns from several studies comparing x and y groups. I've also computed the ratio of the means for each ...
zephryl's user avatar
  • 190
1 vote
0 answers
41 views

Ordinal logistic regression with aggregated data

I'm wondering if it is feasible to conduct a ordinal logistic regression on the following dataset. I want to investigate if there is a correlation between the alcohol outlet density (as counts) and ...
Gian Gyr's user avatar
1 vote
0 answers
22 views

Best way to analyse two experiments

I'm looking for help/indication on approaching a data integration problem. I have a dose-response curve that is described by a log curve. Because of measurement difficulties, the dose and response can'...
Lu_Ste's user avatar
  • 11
0 votes
0 answers
47 views

How to aggregate standard deviations of subsets robustly?

Assume I know the sample standard deviations $s_i, i=1..N$ of $N$ subsets of data. I can assume that the subsets have the same size (usually $S=2$) and the same underlying real standard deviation $\...
user829755's user avatar
0 votes
0 answers
48 views

Comparing means on individual observation level data or aggregated daily level data for two periods

I have data on purchases on individual customer level. The customers are segmented into different groups, like region. I want to compare sales from different periods. Should I use the daily aggregated ...
sp09_81's user avatar
1 vote
0 answers
47 views

Is kruskall wallis appropriate here?

I have collected campaign finance data about donations individuals have made to 3 groups (A, B, C) between 2015 and 2020. Each donor has only donated to one group but some donors have donated ...
user3020416's user avatar
2 votes
1 answer
98 views

Aggregated time series or regression approach?

The setup I describe below is analogous to my actual problem. Problem: I have millions of individuals in my dataset and for each individual, I have certain stats over time. I.e. for one individual Joe,...
Froozle's user avatar
  • 75
0 votes
0 answers
92 views

Loss of variance in aggregated data?

I have a nested dataset where information on individual workplace characteristics is available on the case level, and data on recorded sick leave on the group level. 7k individuals are nested within ...
ym.87's user avatar
  • 1
1 vote
0 answers
108 views

Can I use logistic regression from individual-level survey data to make predictions at an aggregate level?

I have a logistic regression model that predicts the likelihood of an individual getting PTSD after a flood based on their race and gender. The regression is based on a survey of individual flood ...
bashtown's user avatar
0 votes
1 answer
293 views

weighted average in collaborative filtering

I studied on collaborative filtering recently and found that for ratings at last most methods applied were weighted average, no matter what they had proposed, similarity, time, etc. Thus I am ...
Maul Seil's user avatar
0 votes
1 answer
34 views

Calculating average for each individual or not?

I have a question about how I should organize my data before doing some statistical analyses. In my data, I have a sample of rats for which the level of tear stains and body lesions is measured. They ...
balzy's user avatar
  • 129
1 vote
2 answers
797 views

How to solve the problem of having sparse data that would become too small when aggregated?

I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
Traveling Salesman's user avatar
1 vote
1 answer
41 views

Aggregating and averaging time series data to have same nrow per group

I suspect the solution will be pretty straightforward but I'm struggling to figure it out. The problem: I have an experimental design with several video stimuli of different time lengths (between 90s ...
Luminosa's user avatar
  • 101
0 votes
0 answers
16 views

Handling time series data with same start and end datetime but different vector lengths

I am dealing with time series data with 15 different features from a machine. The thing is that the different feature datasets that have been collected have been done at different frequecies. That is ...
Bill Henson's user avatar
1 vote
0 answers
28 views

How does aggregation across time bring about feedback?

On page 11 Pankratz says: "Sometimes there is sound theoretical reasoning in favor of a one-way relationship from the inputs to the output (no feedback). But even then it is important to have a ...
ColorStatistics's user avatar
0 votes
0 answers
142 views

Intuitive explanation of Choquet Integral for data aggregation

I've recently stumbled upon this python package that implements Choquet integral as a way of aggregating data. Does anyone have an intuitive way of explaining how does this integral work in this case?
F.Hand's user avatar
  • 1
0 votes
1 answer
113 views

Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
GaussianBlur's user avatar
1 vote
0 answers
25 views

Staggered Average and Standard Deviation of incomplete dataset

I have an AxBxC matrix of measurements, from which an arbitrarily-sized-and-distributed number of measurements have been deemed invalid and removed (i.e. that matrix contains a mostly random ...
SysEng91's user avatar
4 votes
1 answer
54 views

What type of statistical inferences are possible on aggregated tables?

I have a table with the following format CountH SubcountH1 SubcountH2 CountK... VarA SubA1 SubA2 SubA3 VarB SubB1 SubB2 . . . The table is filled with ...
William M.'s user avatar
2 votes
0 answers
211 views

How to calculate aggregate standard deviation for composite scores in a meta-analysis

I wonder if someone would be kind and able to provide guidance/suggestions on a stats issue that myself and colleagues are trying to handle while performing a systematic review and meta-analysis. In ...
Antonio Dello Iacono's user avatar
0 votes
0 answers
54 views

Aggregating forecasts and their standard errors over time periods

Good day all, I have an ARIMA model which gives me a weekly forecast and standard error for each period. If I were to aggregate the forecasts into monthly time-windows (for simplicity sake, let's ...
ErrHuman's user avatar
  • 101
1 vote
0 answers
29 views

Combined multiple within-group effect sizes

I'm conducting a meta-analysis on observational studies examined the association of age with different drug groups prescription and in order to combine the effect sizes of all drug groups together, a ...
Fatema 's user avatar
1 vote
1 answer
210 views

Is there a right way to aggregate timeseries data for anomaly detection and forecasting?

Hello Cross Validated Users, I have had a question on my mind for a long time whenever I work with timeseries but I have never quite found the right answer. Here is the thing: when creating an ...
ldoe's user avatar
  • 111
0 votes
1 answer
74 views

How can I verify the correlation between appetizers and main meals

I have a data set of users. that looks something like this. user_id appetizer main_meal 185 30 120 142 8 56 34 0 200 etc. etc. .. there are other columns but I don't believe it's relevant for my ...
Nick's user avatar
  • 123
0 votes
0 answers
44 views

Implications of using the mean of the max as a predictor in regression

What are the implications of using a predictor which has been aggregated twice (the mean of a set of max's) as a variable in a regression analysis? I'm performing a regression on the panel dataset: <...
codemachino's user avatar
0 votes
0 answers
54 views

Obtaining better estimates when you know that a set of input variables are independently and identically distributed

Suppose I have a family of random variables $$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$ and I know how to sample $SomeDistribution_i$ for any $i$. Suppose I also define a random variable $Y = f(...
user118967's user avatar
0 votes
1 answer
34 views

How to assess performance when it greatly varies from year to year?

We have an annual performance metric for a number of educational institutions. The figure for any one institution varies a lot from year to year, as it depends very greatly on the student cohort, but ...
HugheyDrambuie's user avatar
1 vote
0 answers
72 views

How to convert high to low frequency data

I have data for a macroeconomic variable, and it has monthly frequency but I need to work with it in quarters. Is there a statistical method to perform this frequency conversion that ensures the least ...
manifold's user avatar
  • 141
1 vote
1 answer
24 views

Does autocorrelation at granular intervals still exist at aggregated intervals?

I have a theoretical question autocorrelation at different sized intervals. Suppose I have a dataset at minutely intervals, that follows an AR(2) process and then drops into white noise at lags > 2....
Hamish Gibson's user avatar
0 votes
1 answer
29 views

Composite Indicator - Negative Correlation Issue

I'm trying to build a composite indicator that aims to measure poverty. I'm planning to aggregate a series of variables using PCA. However I have a doubt, all the variables I want to include are ...
Gigi39's user avatar
  • 13
1 vote
0 answers
69 views

Training Individual-Level Predictor using Distribution of Group-level Data

I have a problem in which I'm looking to train an individual-level predictor for outcomes. I have information on individual-level covariates, but I don't have individual-level outcome variables. ...
cowabunga's user avatar
2 votes
1 answer
945 views

Level of Aggregation for Event Study (Staggered DiD) Analysis

This might be a bit basic, but I'm struggling to determine the ideal level of aggregation of the data for an Event Study analysis. I wish to study if a specific policy affected fertility in certain ...
Silas B.'s user avatar
2 votes
3 answers
350 views

Preferred way to sum different time series together (in software)

Is there a canonical/best approach to computationally summing different time series together? What I mean by that is the operation $$ \sum_i{s_i(t)} = S(t) $$ where $s_i(t)$ is the $i$-th time series, ...
Michele Piccolini's user avatar
2 votes
0 answers
134 views

What is the "appropriate" way to aggregate different proportions?

I've found a lot of ways -- arithmetic mean, geometric mean, harmonic mean -- and I've invented things which have no name (and, likely, deserved none). At the end I just get a number and it's hard to ...
Richard Rast's user avatar
3 votes
1 answer
320 views

Calculating an aggregated score

I'm out of my element when dealing with statistics, so I hope you'll be able to offer me some guidance. I'm working on a project where students will apply for scholarships, and then a panel of people (...
Brian Brock's user avatar

1
2 3 4 5 6