What is exactly meant by a "data set"? Is it just the aggregation of data points? Or is it the representation of data points for different elements in a tabular format arranged with values of the different variables? How is it different from raw data?
 A: In my experience, "dataset" (or "data set") is an informal term that refers to a collection of data. Generally a dataset contains more than one variable and concerns a single topic; it's likely to concern a single sample.
A mistake I often see writers of Cross Validated questions make is using "dataset" as a synonym for "variable" or "vector".
A: I think that Wikipedia does a decent job at defining it:

Most commonly a data set corresponds to the contents of a single
  database table, or a single statistical data matrix, where every
  column of the table represents a particular variable, and each row
  corresponds to a given member of the data set in question. The data
  set lists values for each of the variables, such as height and weight
  of an object, for each member of the data set. Each value is known as
  a datum. The data set may comprise data for one or more members,
  corresponding to the number of rows.
The term data set may also be used more loosely, to refer to the data
  in a collection of closely related tables, corresponding to a
  particular experiment or event. An example of this type is the data
  sets collected by space agencies performing experiments with
  instruments aboard space probes.
In the open data discipline, dataset is the unit to measure the
  information released in a public open data repository. The European
  Open Data portal aggregates more than half a million datasets. In this
  field other definitions have been proposed but currently there is not
  an official one. Some other issues (real-time data sources,
  non-relational datasets, etc.) increases the difficulty to reach a
  consensus about it.

As you can see, the term is somewhat vague.
A: I think you might need to define data point before you can define data set: why is one primitive and not needing definition, but not vice versa? 
At least two definitions make sense to me: 


*

*One or more observations (cases, records, rows) for one or more variables (fields. columns). 

*Whatever is stored as data within a file readable by a program of choice.  
Tabular layout is common but I don't think it's part of any definition; how the data are stored can be practically important, naturally. 
P.S. The word "format" is so overloaded that to me it's best avoided unless specified unambiguously. I've seen it used for 


*

*General or specific text or binary file format 

*Data structure, e.g. tabular or other 

*Data storage or variable types, e.g. bit, integer, real, character

*Display format controlling presentation, e.g. details on number of decimal places; decimal, hexadecimal or binary display. 
A: There are already some good answers here and I don't think I can penetrate any deeper than Nick Cox or Franck Dernoncourt the issue of whether "dataset" refers to the conceptual collection of related data, or to the particular arrangement of those data e.g. into a table/matrix or a computer-readable file. Franck's extract mentions edge cases like continuously-collected data, or data spread across several tables, which are worth bearing in mind if you assumed there was going to be a simple definition. (Not all statistics software can handle it, but it is very easy to imagine a case where data is stored in a relational database with multiple tables. Is the entire database a single "dataset"?)
One thing I will add though is that datasets aren't generally sets, in the mathematical sense! Sensu stricto either a set contains an object or it doesn't, but can't contain more than one copy of that object. If I roll a die eight times and score 1, 4, 3, 5, 5, 4, 6, 4 then the set of scores rolled is just {1, 3, 4, 5, 6}. Note that the elements could be in any order, I've just written them ascending in value but the set {5, 4, 1, 6, 3} is mathematically equal to it, for instance. This isn't what we usually mean by a dataset though!
A multiset (or bag) allows entries to be repeated, e.g. {1, 4, 3, 5, 5, 4, 6, 4} though note this still doesn't include a sense of order, so is equal to {1, 3, 4, 4, 4, 5, 5, 6}. Perhaps the "set" in "dataset" might best be read as "multiset".
Moreover, if you want order to be preserved, you might instead use a vector: (1, 4, 3, 5, 5, 4, 6, 4) is not the same as (1, 3, 4, 4, 4, 5, 5, 6). The ordering gives us an index which can serve as a kind of identifier — it tells us, for instance, "which four is which?" — and which often serves a purpose for recording observations in their natural temporal or geographic order. When one sees formulae such as $\bar x = \frac{1}{n} \sum_{i=1}^n x_i$ this sort of indexing scheme is assumed. In the context of a set or multiset, what would $x_1$ or $x_2$ mean, given that we can't distinguish a "first" or "second" element due to the lack of ordering?
But vectors are only for recording one variable - for several, it may be more convenient to use a matrix to tabulate with order preserved. For more sophisticated situations such as measuring a property of a three-dimensional grid of voxels over time, you might even move up to arranging the data in a tensor (see e.g. this question).
But note that conceptually a multiset may suffice in most simple situations, even if it's inconvenient for practical purposes. If I tossed a coin simultaneously with rolling the die, and wanted to record the two results together, then I could use a multiset like {(1, H), (3, T), (4, H), (4, H), (4, T), (5, H), (5, T), (6, T)} instead of a matrix. An ordinary set will not suffice, as it wouldn't count the multiplicity of the (4, H), for instance.
