6
$\begingroup$

After reading a recent paper by Hadley (link), I got to thinking about whether what we'd refer to as tidy data changes by application. For example, consider a sample dataset:

Food item | Carbohydrates | Fat
F1        | 10            | 12
F2        | 16            | 19
F3        | 29            | 30
F4        | 11            | 28
F5        | 23            | 21

For visualization, a tidy way to represent this would be to create a column called Category that takes values Fat and Calories, giving a 10x2 dimensional data set. A format like this (long form) is useful in visualization, for e.g. in Tableau (see discussion here).

Food item | Value | Category
F1        | 10    | Carbohydrates
F2        | 16    | Carbohydrates
F3        | 29    | Carbohydrates
F4        | 11    | Carbohydrates
F5        | 23    | Carbohydrates
F1        | 12    | Fat
F2        | 19    | Fat
F3        | 30    | Fat
F4        | 28    | Fat
F5        | 21    | Fat

However, let's say I add an observational column named healthy, which takes values yes and no. Now, I am interested in the classification problem of whether a food item is healthy or not.

Food item | Calories | Fat | Healthy
F1        | 10       | 12  | yes
F2        | 16       | 19  | yes
F3        | 29       | 30  | no
F4        | 11       | 28  | no
F5        | 23       | 21  | no

From Hadley's discussion, R models always take tidy inputs. But, in my experience, the input to a model in R would be more intuitive in the format above and not the "tidy" format from earlier (where it would take factor levels of the variable category, complicating interactions etc.). Also, since fat and carbohydrates are two attributes of the same observation, it is reasonable that they appear in one row (similar to the example about the left and right hand in the paper).

So, for the classification problem, does the tidy data format now change? Or was it always as such and the visualization scenario just an artifact of Tableau?

$\endgroup$
  • 2
    $\begingroup$ R has had builtin methods to change between long format and wide format for decades before Hadley Wickham started ggplot2 because there were always situations in which one or the other were preferable. The hottest thing in graphics before ggplot2 was lattice which takes data in wide format IIRC. Just because something is more to the taste of Hadley Wickham, it does not mean, it is more tidy. If the programmers of most regression algorithms in R and of lattice and of ... expect data in wide format, then Tableau is not so special in that opinion. $\endgroup$ – Bernhard Feb 2 '17 at 7:52
  • $\begingroup$ Ah. I believe the notion of wide form and long form predates the concept of 'tidy data', the latter being introduced by Wickham much later. My question pertained to that concept, specifically. To me, it seemed to be a moniker for long form in most cases and I was (still am) confused about how the concept makes it easier to think of regression/classification problems. $\endgroup$ – Naumz Feb 3 '17 at 2:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.