I study deep learning and the one of the major problem I face is I can't imagine shape of higher order tensors in my head.
for instance - A 2d tensor - (x,y) is a rectangle with x,y along its length and breadth. 3d tensor - x,y,z is a 3d figure After that, I just go into brain-freeze, how to think about them Is there a way which more experienced data-scientist use to think about them?

  • $\begingroup$ you can't imagine unimaginable. you need to get used to working with unimaginable, with abstractions $\endgroup$
    – Aksakal
    Commented Dec 11, 2019 at 21:56
  • 2
    $\begingroup$ Physicists describe and work with tensors as "objects with indices that transform in the right way under a change of reference frame." What this means in practice is that they can do amazing calculations with high-rank tensors without having to visualize them at all. $\endgroup$
    – whuber
    Commented Dec 11, 2019 at 22:23

2 Answers 2


You can think of it as descripbed below.

enter image description here

But most of the times I acturally think of it more like the acturally implimentation. Where any array is stored a with dimensions X,Y,Z stored as a flat list with length X* Y* Z. Where the entry a[x,y,z] is element x* Y* Z+y*Z+z. This is pretty useful when think of cache optimization. Take a look at how multidimensional arrays are stored and accessed in memory.


Peter has an excellent way of thinking about tensors in terms of geometric objects. If you need to know how each entry in a tensor is structured that is perfect. Possibly this is fairly bulky to think about so you might consider other ways that people have approached higher level tensors. I would propose something like Penrose's notation. This allows you to represent these more complicated ideas without needing to think about cubes of cubes etc. If you want to demonstrate computation and abstract these issues away this would work.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.