In a SVM you are searching for two things: a hyperplane with the largest minimum margin, and a hyperplane that correctly separates as many instances as possible. The problem is that you will not always be able to get both things. The c parameter determines how great your desire is for the latter.
I have drawn a small example below to illustrate this. To the left you have a low c which gives you a pretty large minimum margin (purple). However, this requires that we neglect the blue circle outlier that we have failed to classify correct. On the right you have a high c. Now you will not neglect the outlier and thus end up with a much smaller margin.

![enter image description here][1]

So which of these classifiers are the best? That depends on what the future data you will predict looks like, and most often you don't know that of course.
If the future data looks like this:

![large c is best][3]
then the classifier learned using a large c value is best.

On the other hand, if the future data looks like this:

![low c is best][4]
then the classifier learned using a low c value is best.


  [1]: https://i.sstatic.net/GbW5S.png
  [3]: https://i.sstatic.net/07jiy.png
  [4]: https://i.sstatic.net/jfJ9G.png