Embed Service Discovery

When you have an application that needs to talk to a service, the tool you use for service discovery needs to perform the following tasks:

• Manage a registry of services containing info such as their IPs and ports;
• Help services find other services using the registry;
• Health check service instances and remove them if they’re not well; and
• Deregister services when they go offline.

Historically, people who’ve built distributed services have depended on separate, stand-alone services for service discovery (such as Consul, ZooKeeper, and Etcd). In this architecture, users of your service run two clusters: one for your service and one for your service discovery. The benefit of using a service-discovery service is that you don’t have to build service discovery yourself. The downside to using such a service, from your users’ standpoint, is that they have to learn, launch, and operate an extra service’s cluster. So using a stand-alone service for discovery removes the burden from your shoulders and puts it on your users’. That means many users won’t use your service because the burden is too much for them, and users who do take it on won’t recommend your service to others as often or as highly.

So why did people who built distributed services use stand-alone service-discovery services, and why did their users put up with the extra burden? Because neither had much of a choice. The people building distributed services didn’t have the libraries they needed to embed service discovery into their services, and users didn’t have other options.

Fortunately, times have changed. Today, Gophers have Serf—a library that provides decentralized cluster membership, failure detection, and orchestration that you can use to easily embed service discovery into your distributed services. Hashicorp, the company that created it, uses Serf to power its own service-discovery product, Consul, so you’re in good company.

Using Serf to embed service discovery into your services means that you don’t have to implement service discovery yourself and your users don’t have to run an extra cluster. It’s a win-win.

When to Depend on a Stand-Alone Service-Discovery Solution

You may encounter cases where depending on a stand-alone service for service discovery makes sense—for example, if you need to integrate your service discovery with many platforms. You sink a lot of effort into that kind of work, and that’s likely a poor use of your time when you could just use a service like Consul that provides those integrations. In any case, Serf is always a good place to start. Once you’ve developed your service to solve the core problem it’s targeting and your service is stable or close to it, then you will have a good sense of whether you need to depend on a service-discovery service.

Here are some other benefits of building our service with Serf:

• In the early days of building a service, Serf is faster to set up and build our service against than having to set up a separate service.

• It’s easier to move from Serf to a stand-alone service than to move from a stand-alone service to Serf, so we still have both options open.

• Our service will be easier and more flexible to deploy, making our service more accessible.

So for our service, we’ll use Serf to build service discovery.

Now that we’ve seen the benefits of using Serf, let’s quickly discuss how Serf does its thing.