Questions tagged [aggregation]
Refers to "lumping together" potentially inhomogeneous groups of data.
247
questions
1
vote
0
answers
16
views
Can I use logistic regression from individual-level survey data to make predictions at an aggregate level?
I have a logistic regression model that predicts the likelihood of an individual getting PTSD after a flood based on their race and gender. The regression is based on a survey of individual flood ...
- 11
0
votes
0
answers
18
views
Restructuring behavioral data
For practice purposes, I am trying to analyze a pilot data from my psycholinguistic experiment through SPSS. Since many experiment softwares provide the data in a trial-by-trial format, I have to ...
- 101
0
votes
0
answers
8
views
Sufficient conditions for equivalence between regression on individual or group average data
I have often hear that running a regression on individual or (properly weighted) aggregated data should give the equivalent results, for example in this question or in Mostly Harmless Econometrics, ...
- 253
0
votes
0
answers
45
views
Using and Reporting ICC for Aggregated Data About Groups of People
I've designed a survey that consists of multiple questionnaires with Likert-scale questions, each of which targets a specific variable in my research model.
The participants of this survey are ...
- 1
0
votes
1
answer
36
views
weighted average in collaborative filtering
I studied on collaborative filtering recently and found that for ratings at last most methods applied were weighted average, no matter what they had proposed, similarity, time, etc.
Thus I am ...
0
votes
0
answers
7
views
Model choice for group-level outcome over time with individual-level predictors
Main question: What is an appropriate model for this?
Similar unanswered question here.
So far, I've just calculated group-level means of the predictor at each timepoint and run an aggregated ...
- 11
0
votes
1
answer
33
views
Calculating average for each individual or not?
I have a question about how I should organize my data before doing some statistical analyses.
In my data, I have a sample of rats for which the level of tear stains and body lesions is measured. They ...
- 121
1
vote
2
answers
171
views
How to solve the problem of having sparse data that would become too small when aggregated?
I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
0
votes
0
answers
16
views
Bootstrapping bootstraps
I am working with data from a software architecture that requires that computation on the underlying sample data be done in batches (memory constraints). These batches may or may not be independent.
...
- 101
1
vote
1
answer
26
views
Aggregating and averaging time series data to have same nrow per group
I suspect the solution will be pretty straightforward but I'm struggling to figure it out.
The problem:
I have an experimental design with several video stimuli of different time lengths (between 90s ...
- 51
0
votes
0
answers
48
views
Cross validation and aggregated data for count models
I know that this question has already been formulated in other posts where there are quite comprehensive answers, I refer especially to these two explanations (thread1, thread2). However I continue to ...
- 51
0
votes
0
answers
12
views
Testing individual effects on aggregate dependent variable
Please help!
How do you test the effect of individual observations on an aggregate dependent variable?
Can HLM or some other multilevel mixed effects model help me test this? I know these models can ...
0
votes
0
answers
30
views
How to calculate a confidence interval for a variable that is based on multiple samples?
Let's say there are multiple (ex: 3) coaches who are each recording the speed of the same sprinter over a constant distance multiple times (ex: 10). In order words, coach 1 will record the speed of ...
- 273
0
votes
0
answers
17
views
How to aggregate time-bucketed values across a set
I am trying to perform some analysis over a dataset of multiple projects. This dataset has multiple months of data for every single project. Because projects occur at different times and have ...
- 111
0
votes
0
answers
12
views
Comparing model fit on data aggregated at different levels
I'm fitting a non-linear model to data aggregated at 5 different levels and want to deduce at which level the model fits the data best, i.e. compare the model fits at the 5 different aggregation ...
- 1
0
votes
0
answers
14
views
Handling time series data with same start and end datetime but different vector lengths
I am dealing with time series data with 15 different features from a machine. The thing is that the different feature datasets that have been collected have been done at different frequecies. That is ...
1
vote
0
answers
24
views
How does aggregation across time bring about feedback?
On page 11 Pankratz says: "Sometimes there is sound theoretical reasoning in favor of a one-way relationship from the inputs to the output (no feedback). But even then it is important to have a ...
- 3,338
0
votes
0
answers
42
views
Intuitive explanation of Choquet Integral for data aggregation
I've recently stumbled upon this python package that implements Choquet integral as a way of aggregating data. Does anyone have an intuitive way of explaining how does this integral work in this case?
- 1
0
votes
0
answers
122
views
How to combine data from different time periods for time-series prediction?
Consider a multivariate time series forecasting task where I have two datasets A and B. A goes from 1960 to 2020 and B goes from 2010 to 2020. There is a feature f ...
- 25
0
votes
1
answer
64
views
Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two
I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
1
vote
0
answers
17
views
Staggered Average and Standard Deviation of incomplete dataset
I have an AxBxC matrix of measurements, from which an arbitrarily-sized-and-distributed number of measurements have been deemed invalid and removed (i.e. that matrix contains a mostly random ...
- 9
4
votes
1
answer
35
views
What type of statistical inferences are possible on aggregated tables?
I have a table with the following format
CountH SubcountH1 SubcountH2 CountK...
VarA
SubA1
SubA2
SubA3
VarB
SubB1
SubB2
.
.
.
The table is filled with ...
- 400
0
votes
0
answers
22
views
Aggregation followed by categorization or vice versa?
For unsupervised classification purposes using several variables, I need to categorize one continuous variable into two classes, particularly, using Z-scores. The issue is that the continuous data ...
2
votes
0
answers
66
views
How to calculate aggregate standard deviation for composite scores in a meta-analysis
I wonder if someone would be kind and able to provide guidance/suggestions on a stats issue that myself and colleagues are trying to handle while performing a systematic review and meta-analysis.
In ...
0
votes
0
answers
34
views
Aggregating forecasts and their standard errors over time periods
Good day all,
I have an ARIMA model which gives me a weekly forecast and standard error for each period.
If I were to aggregate the forecasts into monthly time-windows (for simplicity sake, let's ...
- 101
1
vote
0
answers
14
views
Combined multiple within-group effect sizes
I'm conducting a meta-analysis on observational studies examined the association of age with different drug groups prescription and in order to combine the effect sizes of all drug groups together, a ...
- 41
0
votes
1
answer
61
views
Is there a right way to aggregate timeseries data for anomaly detection and forecasting?
Hello Cross Validated Users,
I have had a question on my mind for a long time whenever I work with timeseries but I have never quite found the right answer.
Here is the thing: when creating an ...
- 101
0
votes
1
answer
52
views
How can I verify the correlation between appetizers and main meals
I have a data set of users. that looks something like this.
user_id
appetizer
main_meal
185
30
120
142
8
56
34
0
200
etc.
etc.
..
there are other columns but I don't believe it's relevant for my ...
- 123
0
votes
0
answers
27
views
Implications of using the mean of the max as a predictor in regression
What are the implications of using a predictor which has been aggregated twice (the mean of a set of max's) as a variable in a regression analysis?
I'm performing a regression on the panel dataset:
<...
- 35
0
votes
0
answers
8
views
Use aggregate data with a linear regression
I am working on a project for my own portfolio regarding movies dataset.
I want to go further on the idea that there is a pretty high correlation between Budget and Revenue. Alex the analyst made a ...
- 1
0
votes
0
answers
53
views
Obtaining better estimates when you know that a set of input variables are independently and identically distributed
Suppose I have a family of random variables
$$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$
and I know how to sample $SomeDistribution_i$ for any $i$.
Suppose I also define a random variable $Y = f(...
- 379
0
votes
1
answer
25
views
How to assess performance when it greatly varies from year to year?
We have an annual performance metric for a number of educational institutions. The figure for any one institution varies a lot from year to year, as it depends very greatly on the student cohort, but ...
1
vote
0
answers
31
views
How to convert high to low frequency data
I have data for a macroeconomic variable, and it has monthly frequency but I need to work with it in quarters. Is there a statistical method to perform this frequency conversion that ensures the least ...
- 141
1
vote
1
answer
22
views
Does autocorrelation at granular intervals still exist at aggregated intervals?
I have a theoretical question autocorrelation at different sized intervals. Suppose I have a dataset at minutely intervals, that follows an AR(2) process and then drops into white noise at lags > 2....
- 131
0
votes
1
answer
16
views
Composite Indicator - Negative Correlation Issue
I'm trying to build a composite indicator that aims to measure poverty. I'm planning to aggregate a series of variables using PCA. However I have a doubt, all the variables I want to include are ...
- 13
1
vote
0
answers
39
views
Training Individual-Level Predictor using Distribution of Group-level Data
I have a problem in which I'm looking to train an individual-level predictor for outcomes. I have information on individual-level covariates, but I don't have individual-level outcome variables. ...
- 11
2
votes
1
answer
342
views
Level of Aggregation for Event Study (Staggered DiD) Analysis
This might be a bit basic, but I'm struggling to determine the ideal level of aggregation of the data for an Event Study analysis.
I wish to study if a specific policy affected fertility in certain ...
- 21
1
vote
1
answer
54
views
Preferred way to sum different time series together (in software)
Is there a canonical/best approach to computationally summing different time series together?
What I mean by that is the operation
$$
\sum_i{s_i(t)} = S(t)
$$
where $s_i(t)$ is the $i$-th time series, ...
2
votes
0
answers
35
views
What is the "appropriate" way to aggregate different proportions?
I've found a lot of ways -- arithmetic mean, geometric mean, harmonic mean -- and I've invented things which have no name (and, likely, deserved none). At the end I just get a number and it's hard to ...
- 335
3
votes
1
answer
85
views
Calculating an aggregated score
I'm out of my element when dealing with statistics, so I hope you'll be able to offer me some guidance.
I'm working on a project where students will apply for scholarships, and then a panel of people (...
- 131
1
vote
0
answers
28
views
The correct way to interpret aggregated-level correlation
One keeps seeing comments from the statisticians that say researchers interpret the correlations based on aggregated data falsely. This made me wonder, well how does one correctly interpret it then?
...
- 11
0
votes
0
answers
15
views
Issues when using independent variables from two levels for prediction on single level
This is my first question here and I want to make sure I am giving as much relevant background as possible, so please bear with me!
I am analysing the factors influencing commute mode choice in ...
- 1
0
votes
0
answers
102
views
Negative binominal regression with aggregated data in Stata
A question concerning the effect of aggregation level on negative binominal regression estimates. Example with Stata.
Say I have the following data (showing first rows of an an example data):
...
- 1
1
vote
0
answers
27
views
Creating an interaction effect with data that is aggregated to a daily level
This is my first question so apologies if I am not providing enough information or context. I am working with a dataset where my dependent and most independent variables are at a daily level, but two ...
- 11
0
votes
1
answer
116
views
How to calculate annual return AR(1) coefficient from monthly return AR(1) coefficient?
I have a time-series data of $m$ months stock returns ($r_t$) that I assume satisfies the $AR(1)$ process:
$r_{t+1} = \rho r_t + \epsilon_{t+1}$ for $t = 1,2,3,...,m-1$.
Now I cumulate every 12 ...
- 13
0
votes
0
answers
47
views
How we can aggregate prediction from a forecast?
Let's say that I have a time series forecast of form:
May 104.4
June 103.5
July 107.4
September 109.2
Afterwards I would like to compute an aggregated forecast for the period. Can I just compute ...
- 277
0
votes
1
answer
523
views
SNAIVE METHOD getting same result for different estimators (BU, OLS and STRUC) on THIEF package in R (TSAGGREGATES + RECONCILETHIEF)
I'm stucked and don't know how to solve that problem using thief package in R (actually tsaggragates + reconcile thief) for SNAIVE METHOD. Does anyone can check it and see what is wrong with it? If it'...
0
votes
1
answer
44
views
What are the drawbacks of aggregating regression results if error decreases?
I am trying to predict Total National Sales in dollars for a major retailer. However, the data I'm using exists at the state level, and includes several features around specific brands and foot ...
- 203
0
votes
1
answer
40
views
Aggregation Estimation Issues
I'm analyzing the effect of a law enforcement measure over time on reported violence for districts in a city. If I consider the city as a whole I have access to a lot of possible control variables (e....
- 1
2
votes
1
answer
333
views
Looking for R's way of doing Stata's svy tabulate with "tab" argument [closed]
Stata's svy command allows to get standard error estimates from two way contingency tables and the tab(varname) option makes it ...
- 135