When using an orchestrator, one tells it in a declarative way how one wants it to run a given application or application service. We learned what declarative versus imperative means in Chapter 8, Docker Compose. Part of this declarative way of describing the application service we want to run is elements such as which container image to use, how many instances to run of this service, which ports to open, and more. This declaration of the properties of our application service is what we call the desired state.
So, when we now tell the orchestrator the first time to create such a new application service based on the declaration, then the orchestrator makes sure to schedule as many containers in the cluster as requested. If the container image is not yet available on the target nodes of the cluster where the containers are supposed to run, then the scheduler makes sure they're downloaded from the image registry first. Next, the containers are started with all the settings, such as networks to which to attach, or ports to expose. The orchestrator works as hard as it can to exactly match in reality in the cluster what it got in our declaration.
Once our service is up and running as requested, that is, it is running in the desired state, then the orchestrator continues to monitor it. Each time the orchestrator discovers a discrepancy between the actual state of the service and its desired state, it again tries its best to reconcile the desired state.
What could such a discrepancy between the actual and desired states of an application service be? Well, let's say one of the replicas of the service, that is, one of the containers, crashes due to, say, a bug, then the orchestrator will discover that the actual state differs from the desired state in the number of replicas: there is one replica missing. The orchestrator will immediately schedule a new instance to another cluster node, which replaces the crashed instance. Another discrepancy could be that there are too many instances of the application service running, if the service has been scaled down. In this case, the orchestrator will just randomly kill as many instances as needed to achieve parity between the actual and the desired number of instances. Another discrepancy could be when the orchestrator discovers that there is an instance of the application service running a wrong (maybe old) version of the underlying container image. By now, you should get the picture, right?
Thus, instead of us actively monitoring our application's services running in the cluster and correcting any deviation from the desired state, we delegate this tedious task to the orchestrator. This works very well, if we use a declarative and not an imperative way of describing the desired state of our application services.