1
$\begingroup$

I am a complete newcomer to the field of machine learning. I do have a lot of experience in computer programming, but nothing related to ML.

My question is whether or not ML would be a good approach to isolate portions of a source code file that contain references to a software license. For example, often times a source file will include information at the top of the file describing the code's license, as well as the file's author and copyright info, such as this:

// Copyright (C) 2007-2015 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the terms
// of the GNU General Public License as published by the Free Software
// Foundation; either version 3, or (at your option) any later
// version.

// This library is distributed in the hope that it will be useful, but
// WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// General Public License for more details.

// Under Section 7 of GPL version 3, you are granted additional
// permissions described in the GCC Runtime Library Exception, version
// 3.1, as published by the Free Software Foundation.

// You should have received a copy of the GNU General Public License and
// a copy of the GCC Runtime Library Exception along with this program;
// see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
// <http://www.gnu.org/licenses/>.

This would be an example of a "GPL-3.0+" license. What I would like to do is use ML to isolate and extract this portion of text from the entire source file.

I have access to an existing corpus of such text snippets along with their license classification, so my initial thought is that this would be a supervised learning problem using a classifier of some kind. From what I've read, I also thought that the text can be turned into a vector representation using the bag of words approach, possibly using bigrams instead of single words to preserve some positional information.

A couple of specific questions I have about this approach are:

  1. There are probably 50 or so different kinds of licenses. Is this too many categories for a classifier?
  2. For each type of license, I'm guessing there may be about 5 to 50 examples of the license that could be used for training. Will this be enough data, or are many more examples needed? (Or will this approach never work well no matter how much data is available)?

Like I said, I don't have any experience with LM, so I'd appreciate any thoughts or advice on whether or not this approach makes sense at all for this type of problem.

One more bit of background info: I'm currently using Fossology to flag files that may contain licenses. But as far as I'm aware, Fossology doesn't use ML, it just uses keywords and regular expressions to find candidate files that may contain a certain license.

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ But keywords are totally enough to identify the licenses, aren't they? Licenses have standard forms and standard names, there is no "noise". Moreover, if someone uses modified form of a license, then from legal point of view it's a different license. Maybe I'm wrong, but using ML in here sounds like an overkill. $\endgroup$ – Tim♦ Jun 2 '17 at 17:00
  • $\begingroup$ It's true that the licenses have a standard form, but the license references can be quite a bit different from file to file. That's where keywords come in, but the problem is that they can find false positives. There's also the question of how to extract the relevant text. You may be right that ML is overkill, maybe the best approach is to enhance keyword/regex searching. I'm just trying to explore additional options at this point. $\endgroup$ – Travis Jun 2 '17 at 17:17
  • $\begingroup$ ML is an overkill for this problem. Simple pattern matching will do the job. $\endgroup$ – Aksakal Jun 2 '17 at 17:53
  • $\begingroup$ I'm already using Fossology, which does use simple pattern matching. It "does the job", but it doesn't always do it that well. I guess I'm asking if ML could do a better job. $\endgroup$ – Travis Jun 2 '17 at 18:11
2
$\begingroup$

Using machine learning in here sounds like an overkill. Doing this using ML would be quite complicated: you would need to clean up the source codes, prepare some features, then you'll end up with huge, sparse datasets (literally every word becomes a feature, so you end up with thousands of features) with lots of noise (parts of source code, stopwords, irrelevant comments in the code), next, you will need to apply some methods for text mining, then use ML with possibly lots of tuning since the complicated nature of the data. It sounds like a pretty complicated and time-consuming process. I bet it won't be much better given the workload. Moreover, using ML does not guarantee less false positives.

Since licenses have pretty standard form and there is limited number of them, then, I guess, you can prepare by-hand some rules to classify them and this should not work worse then ML. Even more, you can use expert knowledge to create the rules while ML would create them "blindly" because ML algorithms do not know anything a priori about what licenses are. Hammers are designed to use with nails, you don't need anything more advanced to deal with them.

If fossology does not work for you, then you can always use it with your data and then check the misclassifications to find when and why it fails, so to correct it's mistakes.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks Tim! I appreciate the insight. Would every word necessarily have to be a feature though? I thought there were algorithms for picking an optimal subset of words in some kind of feature reduction step. $\endgroup$ – Travis Jun 2 '17 at 18:28
  • $\begingroup$ BTW, manual (human) review is exactly what's done now to validate the Fossology results. This is a time-consuming and expensive process, and my hope was to reduce the amount of manual review needed, either by using ML or some other method. $\endgroup$ – Travis Jun 2 '17 at 18:29
  • $\begingroup$ @Travis not every, but still you would start with data containing every word and then you will need to decide what words to leave as a features, this is additional problem to solve. $\endgroup$ – Tim♦ Jun 2 '17 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.