A computational graph is a series of TensorFlow operations, also known as ops, arranged into a graph of nodes. The following two principle steps are used by TensorFlow Core:
- Define a computational graph
- Run the computational graph
Let's try to understand this concept with a very simple example. Let's say that you have a function with two variables, x and y as shown in the following screenshot:
We will use the preceding formula to calculate or to build a computational graph for the actual value of this function when you pass the values 3 and 2 for x and y respectively:
Now, let's build a computational graph for actually getting the result from this computation model:
In the preceding screenshot, we see the values that flow through the computational graph to different nodes in the graph. So, in the first node, the value 3 gets assigned to x and, in the other node, the value 2 gets assigned to y. Now, the value of x flows to an operation node where it gets squared, and the result of that node flows to another operation where it gets multiplied to the value of y. We also have another node, where the value of y gets multiplied by 4. The result of the x and y multiplication node and the result of the y multiplication node flow to the final node, which is the addition node, which gives us the final output 26. So this is essentially how TensorFlow works. What flows between nodes are tensors.
There are other following objects that we use in TensorFlow:
- Session: A session is an object that encapsulates the environment in which operation objects are executed. So, sessions are objects that place operations onto devices such as CPUs or GPUs.
- Placeholders: A placeholder is a promise to provide a value later. These objects are usually used to provide training and testing values in machine learning models.
- Variables: These are objects that are initialized with a value, and that value can change during the execution of the graph. Typically, they are used as trainable variables in machine learning models.
- Constants: Constants are objects whose values never change.
To have a better understanding of these object concepts, let's see an example. First, we will import the required libraries by executing the following code snippet:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
We then define some TensorFlow objects, placeholders, and a constant by executing the following code snippet:
#Placeholders
x = tf.placeholder(tf.float32)
y = tf.placeholder(tf.float32)
c = tf.constant(5)
Here, we define a placeholder called x and another placeholder called y. You have to explicitly give the type of object that you will use in TensorFlow, which we have in our example as float32. We then define a constant, c ,whose value is 5.
After you create these objects, if you try to print them, you will not see the value of the object, but it will show the type of the object as shown in the following screenshot:
Now, let's implement the following function with our placeholders:
We will use the placeholders that we just created to define the different nodes for our graph by executing the following code lines:
square_node = x*x
mult_node = square_node*y
quadruple_node = 4*y
adder_node = mult_node + quadruple_node
Again, if you try to print the values of these objects, you will get the object type and not the values as shown in the following screenshot:
So, to perform the calculations for these objects, you must create a session object and then run all of the objects inside a session:
If you are doing some computations, you don't need to define the computational graph, as TensorFlow will do this behind the scenes. So, let's say that you want to calculate f and we print the value, it will still give the object type. But to actually see the value of f when you perform the computation, we will run the function in a session object again:
There are two ways in which you can run objects in TensorFlow. There are other ways, but these are the basic and most common ways you can run objects. You can use the run() method from a session or you can use the eval() method from the tensor:
As we can see, we created a session using the with statement and ran those two methods inside this statement.
Now, we will build a basic linear model. We will have TensorFlow guess the best values for the b and w parameters shown in the following screenshot:
In the previous equation, the value of w is 5 and b is 1. We will use these values for training and plot the values on a scatter plot:
As you can see, we have the linear relationship between the two values.
We will now initiate the variable objects, w and b ,with the value 0, and they will be our trainable parameters. The placeholders are usually the objects that we use to pass the data, so we will create two placeholders, x and y, and now the linear model will be one of the nodes in our computational graph. Then, we will define a loss function, which will be used by the optimizer to actually change the values of our variable. Every time we run the training operation, the optimizer will adjust the values of w and b in order to minimize the loss. We will then initialize the variables and create a session to run the init initializer node as shown in the following screenshot:
Now, we can start training our machine learning model. We will run the training operation 20 times, which will make corrections to our values of w and b to minimize the loss:
As we see, after the first iteration, the optimizer corrected the values of w and b, which is also carried out in every iteration.
We can also do this using some linear algebra, but remember that the goal of machine learning is to actually learn the parameters from the data, and in this case, we have run our first machine learning model using TensorFlow.