I am trying to figure out how to properly do regression analysis on a data set from a peer-review survey where individuals in the survey have an unequal number of responses. Below is description of my actual problem.
Consider a population consisting of $N$ individuals. You send a survey out to all $N$ individuals asking them to rank the quality of everyone (including themselves) in that population wrt some property based on a scale from NR (not rated) to 2,3,4,5. The NR responses are not counted. Suppose that $n$ of the $N$ individuals respond.
Some individuals in this population are of very high quality and so receive rankings from almost everyone in the population that respond (i.e. most of the $n$ responses are not NR). Conversely, some of the individuals are not of high quality and do not receive many rankings. For individual $i$, let $S_i$ denote the sum of all the rankings from the surveys received which are not NR and let $n_i$ denote the number of surveys received for individual $i$ which do not have NR rankings, from which we compute its overall score as $r_i$ = $S_i/n_i$.
Suppose you $only$ know $r_i$ for each individual and do $not$ have access to the original data (and so you do not know $n_i$ for each individual) and want to perform a regression analysis on the scores for some given set of covariates. My question is: how can one compensate or take into account the differences in the number of responses? Would weighted least squares naturally take into account that the individuals with more NR responses should "fit" the data better?
Thank you, Matt