1
$\begingroup$

I'm working on a public procurement dataset where I have information on all the participants that presented offers in 358 tenders. I'm analysing relationships between all the companies of the dataset (1242). I'm running a logistic network regression that allow to predict a relationship knowing another. In my case, for example, I'm interested in understanding the extent to which companies that frequently submit the same offer in a tender are also part of the same cartel (I have information on 8 different colluding cartels active in these tenders).

For each dyad (pair of companies) I'm calculating:

a) the number of times they participated together, and b) the number of times they submitted exactly the same offer.

I am unsure about the b) measure. When I calculate it, I obviously get missing values everytime two companies did not participate together in a tender and so did not have the "opportunity" to submit the same offer. This variable seems to create problems in regression because of the many missing values (85% of the dyads-observations are missing). Consider that the missing are not random and, as I said, I'm perfectly aware that they are missing "by default", because companies that did not participate together, did not bid on the same contract and by consequence could not bid the same offer! I thought that I could fill missing values with "0", thus without requiring this variable to depend to much co-bidding in the same tender. Do you think this approach makes sense or is it a way to force the data too much?

$\endgroup$
  • $\begingroup$ How many missing values are there and how many different non-zero values are there in that "number of times" variable? Can you add counts (approximate would be fine) to the question? I have an idea, potentially. $\endgroup$ – AlexK Apr 6 at 6:01
  • $\begingroup$ Many thanks AlexK. I've added an explanation and the answer to your question $\endgroup$ – Mark Apr 6 at 6:37
  • $\begingroup$ Are you able to add the non-zero values too? When they participated together, was it always one time that they submitted the same offer? Some one time, some twice? What does the data show? $\endgroup$ – AlexK Apr 6 at 7:07
0
$\begingroup$

Clearly missingness is s matter of definition here. 0 seems like a very reasonable value to put in on this particular example.

Whether that is a really good way to construct a feature is another question - in a sense you may be trying to measure how much these companies are similar in terms of what they bid for and how independent their bids are. For the first feature, what happens for large companies that simply do more bids? Perhaps you should standardize by an appropriate total number of bids (perhaps with some penalizing (e.g. add 1 to the denominator) to not overreact to 1 out of 1 = 100%)?

$\endgroup$
0
$\begingroup$

85% missing is a lot. I am not even sure if creating a non-missing value, like 0, for all of those observations would do you much good, because there will be very little variation in that variable and it might serve as simply noise in the model. And similarly, that means that 85% of observations in the other variable have a 0 value, which may also be problematic.

But in any case, if you decide to use some combination of this information in your model, I would not recommend using both variables, because they will essentially measure the same thing. You say changing missing values to 0's would make that variable not depend on co-bidding, but you'll just be copying 0's from one column to another column, so they would not be independent. I would suggest combining the two measures into one categorical or continuous variable (depending on whether the number of times goes up to very large numbers) and assign values to it as follows:

Continuous:

If they did not participate together: 0
If they participated together and submitted the same offer once: 1
If they participated together and submitted the same offer twice: 2
...and so on

or Categorical (that can be converted into 1/0 dummies):

If they did not participate together: A
If they participated together and submitted the same offer once: B
If they participated together and submitted the same offer twice: C
...and so on
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.