Service Discovery

This post deep dives into what is service discovery and what is service registry

Mar 10, 2024

Introduction

At a high level, you might be using the following kinds of architectures:

Monolithic architecture
Microservices architecture
Hybrid (which uses monolithic repos for team development but deploys services in a microservice fashion).

If you consider a microservices architecture hosted in a cloud-based environment in which Service A is calling a downstream service: Service B, then how does Service A know where Service B is hosted(host/port)?

To start with, we can maintain a YAML/JSON file in Service A which has context about where Service B and other services( which Service A needs ) are hosted.

But, Hardcoding the real host/port details is not a good idea.

Why? Tomorrow, if your service B instance goes down because of failures/upgrades or has new instances that are spawned at different hosts and ports, developers will spend most of their time changing the host and port of services in the configuration files. Thus, you can imagine it’s not a scalable solution for the future.

What do we do then?

Enters the solution: Service Discovery

In a monolithic architecture, all important services of the product are tightly integrated and deployed as a single unit. One big repository and thus one big service with possibly multiple instances. Either it’s an on-premise or cloud based architecture, we can balance the load across multiple instances using load-balancer. There's practically no need to find where other services are.

What is Service Discovery?

As the name is intuitive itself, Service Discovery is a way to discover the services in a microservices environment so that maintenance of communication across services becomes easy to manage.

To tackle service discovery in a microservices environment, there are two ways:

Client-Side Discovery Pattern
Server-Side Discovery Pattern

Before we deep dive more into the service discovery patterns mentioned above, let’s understand more about an underlying concept called: Service Registry

What is Service Registry?

The service registry is a special database that maintains the available instances of a microservice. If any new instance comes online or any existing service instance goes down, then the service registry needs to be updated to maintain the current available instances of a microservice. The service registry is updated using the following solutions:

Self-registry: Whenever any service comes up or goes down, each service instance should directly update the service registry about its availability.
Third-party registration: Using the Side Car Pattern, a parallel process along each service instance runs like a daemon that is responsible for maintaining all the available instances in the Service Registry.

The are pros and cons to each service registry method and much more to deep dive into but I’ll save it for some other day.

Client Side Discovery

In client-side discovery, the client side is responsible for first querying the service registry and then fetching the details of the available instances. Then, the client is responsible for load balancing their requests across multiple instances of the downstream service. In this pattern, the client has to do all the heavy lifting.

Client-side Service Discovery (Credits: NGINX article)

Server Side Discovery

In server-side discovery, the server sends a request to the load balancer. The load balancer sends a request to the service registry to fetch the details of available instances of a service. After receiving network locations of all instances of a service, the load balancer routes the requests to available instances using an appropriate load balancing algorithm. In this pattern, all the heavy lifting is extracted from the service and put into action using a load balancer and service registry.

Server-side Service Discovery (Credits: NGINX article)

Pros and Cons

Let’s be aware of the tradeoffs in both types of service discovery patterns.

Infrastructure: The client-side discovery is easy to set up. You just need one service registry and no other infrastructure is required to maintain the communication. On the server side, you need extra infra like one proxy/router/load balancer to support service discovery.
Language dependency: The client-side discovery is language-dependent. If you have multiple services in multiple languages, you need to write a service registry aware HTTP client in each of those services which is essentially duplicate code but in a different language. The server-side discovery is language-agnostic as proxy/router is common across all services.
Tight/Lose Coupling: If any change happens in the service registry, it could affect implementation in all the services in the ecosystem and thus make it tightly coupled. The server-side discovery is loosely coupled as it extracts out the discovering part to the router/proxy which is common across all services.

Service Discovery/Registry examples

Netflix Eureka: It is a service registry. It exposes the POST method (registering an instance), PUT method (updating the availability every 30 seconds), DELETE method (for unavailable instances that don’t register every 30 seconds), and GET method (for fetching the available instances of a service). Eureka is a RESTful service that is primarily used in the AWS cloud for the purpose of discovery. It is also used for load balancing and failover of middle-tier servers by using other tools of Netflix. Overall, Eureka plays a critical role in Netflix's mid-tier infra.
etcd: A highly available, distributed, consistent, key‑value store that is used for shared configuration and service discovery. etcd is written in the Go language and uses Raft as the consensus algorithm.
Apache Zookeeper – A Zookeeper in Kafka acts as a Coordinator service tracking which brokers are active and available. However, this is getting deprecated in favor of a new consensus protocol called Kafka Raft.
consul: Create a central registry that tracks services, updates, and health statuses in real-time.

That’s it, folks for this edition of the newsletter. Please consider liking and sharing with your friends as it motivates me to bring you good content for free. If you think I am doing a decent job, share this article in a nice summary with your network. Connect with me on Linkedin or Twitter for more technical posts in the future!

Book exclusive 1:1 with me here.