3
$\begingroup$

Why do we need tensors in Tensorflow? I mean, if numpy can be used to create multidimensional arrays, which tensors essentially are, why do we bother creating tensors instead of numpy arrays?

$\endgroup$
2
  • $\begingroup$ See also this part of the PyTorch tutorial. $\endgroup$ – erocoar Jul 11 '18 at 13:49
  • $\begingroup$ You may be interested in this project, which allows you to automatically differentiate native Python and numpy code: github.com/HIPS/autograd $\endgroup$ – Lucas Jul 11 '18 at 14:10
7
$\begingroup$

Tensors, as defined by the deep learning software are multidimensional arrays, so if you need only to conduct simple (small-scale) mathematical operations and transformations on the data, then TensorFlow is an overkill. But TensorFlow is much more then this,

  • it implements most of the common building bricks for building the deep learning models,
  • it has the state-of-the-art optimization algorithms,
  • it does automatic differentiation out-of-the-box,
  • it supports GPU training,
  • it has high level (Estimator), middle level (Kears) and low level (core TensorFlow) interfaces depending if you want to program something by hand, or just train a generic model,
  • the probabilistic programming module is developed,
  • there is an ongoing work of building Spark interface for TensorFlow, so it will natively integrate with big data environments,
  • it can be easily integrated in production environment,
  • or the models can be transformed to TensorFlow Light and ran on mobile devices, or translated to JavaScript,

and it has many, many more features.

$\endgroup$
6
  • 1
    $\begingroup$ I don't think this answers the question of why Tensorflow decided to use a new tensor abstraction instead of using numpy arrays. $\endgroup$ – cosmosa May 14 '20 at 18:24
  • $\begingroup$ @cosmosa I’m not sure what kind of answer would you expect? It is different software then numpy and defines its own objects. Tensors are arrays. $\endgroup$ – Tim May 14 '20 at 19:02
  • $\begingroup$ Sure, but numpy is pervasive as a multidimensional array in python. For example pandas , instead of creating a new data structure, just builds on top of numpy. I was also surprised to see TF did not do the same but implemented its data structure from scratch. It's probably a newbie thought from someone who hasn't used TF too much, but the OP had the same question. The other answer seems to address this question better. $\endgroup$ – cosmosa May 15 '20 at 20:17
  • $\begingroup$ @cosmosa TF is not python, it just has python interface, that's all. I didn't check any statistics on it, but if you check the repo, it's mostly C++ code. $\endgroup$ – Tim May 15 '20 at 20:24
  • $\begingroup$ good, then that is a good explanation for why TF did not wrap numpy, because they needed multiple language interfaces for it. $\endgroup$ – cosmosa May 15 '20 at 21:55
6
$\begingroup$

I am assuming that a) you do not question why do we have TensorFlow itself, and understand its value; and b) you only question why wouldn't TF use np.array class instead of creating a new class tf.Tensor.

When you write any kind of extensive framework such as TensorFlow, you end up creating many types and even type hierarchies. Creation of every single one of them could potentially be challenged, of course. However, it doesn't make a sense to do it individually. You have to consider the context of the framework. For instance, sometimes you end up creating almost exact copies of classes from the already existing package. Why not simply import the package? There could be many reasons including that maybe you don't want to drag entire package just to get a few classes from it, so you duplicate them etc. Therefore, from software engineering point of view it is almost easy to discard your question as lacking the context. However, I'll try to answer it because it touches the central data structure of TensorFlow framework.

There are many ways to answer the question, I'll only pick one. Take a look at the following numpy code in Python:

import numpy as np
np.array([[1,2],[3,4]])**2

Here's the output it produces, an element-wise square of the 2x2 array:

array([[ 1,  4],
       [ 9, 16]], dtype=int32)

Next, look at a seemingly equivalent TensorFlow code:

import tensorflow as tf
t = tf.constant([[1,2],[3,4]])
t = t**2
t

Let's look at its output closely and compare to numpy's:

<tf.Tensor 'pow_1:0' shape=(2, 2) dtype=int32>

This is not the square of the array... yet. This tensor is linked to the description of what will be ultimately calculated by the computation graph that we created above. It's a very simple graph, has only a couple of nodes: input array and the square operation. Nevertheless it's a graph. Let's now evaluate it as follows:

sess = tf.Session()
sess.run(t)

Here's the output:

array([[ 1,  4],
       [ 9, 16]], dtype=int32)

Now, this looks like an array that we expected!

We can make this even more explicit with this TF code:

import tensorflow as tf
import numpy as np

x = tf.placeholder(tf.float32, shape=(2,2))
t = x**2
t

Here, we get the Tensor again, and it can't even be calculated eagerly (thanks, @Tim)!

<tf.Tensor 'pow_3:0' shape=(2, 2) dtype=float32>

We need to plug the actual array instead of the placeholder to get a results as follows:

sess = tf.Session()
sess.run(t,feed_dict={x: np.array([[1,2],[3,4]])})

getting the desired output:

array([[  1.,   4.],
       [  9.,  16.]], dtype=float32)

So, this was the long way to say that Tensor class in TensorFlow is a lot more than just a numpy array. It's almost like a variable vs. value comparison in this context.

$\endgroup$
4
  • $\begingroup$ tf.enable_eager_execution() crushes the example ;) $\endgroup$ – Tim Jul 11 '18 at 13:47
  • $\begingroup$ @Tim, I added an uncrushable version of the example $\endgroup$ – Aksakal Jul 11 '18 at 14:10
  • $\begingroup$ Thank you for your help, this has helped me get a better understanding :) $\endgroup$ – Ćepa Jul 12 '18 at 3:04
  • $\begingroup$ Not sure if this answer still stands after tensorflow 2.0 $\endgroup$ – user3282777 Jan 17 at 6:22
0
$\begingroup$

In addition to the other answer, I think a Tensor can also be an operation. In short a Tensor is an abstraction of a data or operation and it represents a node in the computational graph. The graph is the set of calculations to be done by TF in order to train, predict, etc.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.