i'm pretty new to machine learning so i think this might be a realy basic question.
Let's imagine the following scenario: I want to classify projects as either active or inactive. Projects can be defined by number of contributors etc. The goal is to identify projects with a high risk of abandonment by their owner / contributors. A project can be labeled as "inactive" by its owner. I made the following assumption: if someone labeled his project as inactive, its very unlikely that its active. On the other hand, not having the label does not imply its active, because not everyone goes back to label a project inactive after loosing interest. For the sake of this question: lets say that 10% of the data is labaled as inactive, but in reality, 30% of projects are inactive.
At first i thought: ok, i'll just go through all the data and check for the active ones if they are inactive in reality. But, if i do that, why would i need a model anyways? I would base this labelling on something like "no activity in one year, 0 contributors etc.). And now i am confused: is it right that in this scenario, its not okay to label the data myself? Is it only okay to let humans label data on something like images? Is there any scientific literature that i could quote on this topic?
Additional question: would semi-supervised learning be a use case for this? My idea would be to use a supervised method to train a classifier on the 10% labelled data and then use the unlabeled data aswell (like here: http://matpalm.com/semi_supervised_naive_bayes/does_it_do_any_better.html).
But for testing, can i only use projects that are actually labeled (from the 10%)? So, lets say i use 50% of the projects that are labeled as inactive for training the classifier and 50% for testing. I could validate my model with the test data. Now i would use semi-supervised learning in my modell aswell and compare it again. Is it possible that this would get me better results?