Master Service Mesh Architecture with Istio

1. Introduction

Uncover the magic of service mesh architecture and see how it supercharges your distributed systems.

Dive deep into Istio, the service mesh superstar, and learn how it orchestrates your Kubernetes cluster to perfection.

2. What Is a Service Mesh?

Monolithic applications have evolved into smaller, independent services. This shift gained immense popularity with cloud-native computing and microservices architecture. Tools like Docker and Kubernetes accelerated this trend.

Microservices on platforms like Kubernetes offer numerous benefits, but they also introduce complexities. Managing communication between these distributed services requires careful consideration of discovery, routing, retries, and failover.

Additional challenges include security and observability.

These proxies form a mesh network for the services, hence the name. Service mesh controls every aspect of service-to-service communication, enabling it to address the eight fallacies of distributed computing.

4. Istio Unveiled

Istio, an open-source service mesh brainchild of IBM, Google, and Lyft, invisibly enhances distributed applications with traffic management, security, and performance insights.

Flexibly deployed on-premise, in the cloud, within Kubernetes, or on virtual machines, Istio shines with microservices on Kubernetes. Though platform-agnostic, it's a natural fit for containerized environments.

At its core, Istio deploys Envoy proxies as sidecars alongside each microservice:

This proxy network forms Istio's data plane, orchestrated by the control plane:

Istio's adaptability shines through seamless integration with external logging, telemetry, and policy systems.

5. Unraveling Istio's Building Blocks

Istio's architecture is a dynamic duo: the data plane and the control plane. Together, they orchestrate the magic behind Istio's capabilities. Let's dive into the core components powering this exceptional platform.

Get ready to explore the intricate details of Istio's core components.

5.1. The Data Plane: Envoy's Epicenter

Istio's data plane is essentially an amplified version of the Envoy proxy. This open-source marvel handles network intricacies, freeing applications to focus on their core business. Applications interact seamlessly, oblivious to the complex network infrastructure.

At its core, Envoy is a network wizard operating at layers 3 and 4 of the OSI model. It deftly manages connections using a chain of adaptable network filters. Beyond this, Envoy's L7 layer filter empowers it to handle HTTP traffic with finesse. And that's not all - it's a natural with HTTP/2 and gRPC protocols.

Many of Istio's standout features are actually superpowers inherited from Envoy's built-in abilities:

• Traffic Control: Envoy wields granular control over HTTP, gRPC, WebSocket, and TCP traffic, routing it with precision.
• Network Resiliency: Built-in automatic retries, circuit breaking, and fault injection ensure unwavering network reliability.
• Security: Safeguard your communication with robust security policies, access control, and rate limiting.

Envoy's true potential shines when paired with Istio. Its extensibility, powered by WebAssembly, is a game-changer for custom policy enforcement and telemetry. Plus, Istio's Proxy-Wasm sandbox API opens doors to even more Envoy customizations.

5.2. The Control Plane: Istiod's Maestro

Meet istiod, the conductor of Istio's control plane orchestra. This maestro transforms high-level routing rules and traffic management directives into Envoy-friendly configurations, seamlessly distributing them to sidecars.

Remember Istio's earlier architecture with its individual components? Well, to simplify things, these components were merged into the unified istiod. But fear not, the core functionalities remain intact.

At its heart, istiod leverages the same code and APIs as its predecessors. Pilot, for instance, continues to be the maestro of service discovery, translating platform-specific details into a universal language understood by sidecars. This flexibility allows Istio to harmonize with diverse environments like Kubernetes and Virtual Machines.

istiod also takes charge of security, establishing robust service-to-service and end-user authentication with its built-in identity and credential management system. Enforce granular security policies based on service identity with ease. Moreover, istiod functions as a trusted Certificate Authority (CA), issuing certificates to secure communication between services using mutual TLS (MTLS).

6. How Istio Works

We've explored the typical features of a service mesh and dissected Istio's architecture and core components. Now, let's uncover how Istio delivers these features using its core components.

We'll revisit the same feature categories we explored earlier.

6.1. Traffic Management

Istio's traffic management API offers granular control over service mesh traffic. Tailor traffic configurations using these APIs and define API resources with Kubernetes custom resource definitions (CRDs). Key API resources for traffic routing are virtual services and destination rules:

A virtual service dictates how requests are routed to a service within the Istio mesh. It comprises one or more routing rules evaluated sequentially. After virtual service routing, destination rules are applied to control traffic to a destination, such as grouping service instances by version.

6.2. Security

Istio's security foundation is strong identities for every service. Istio agents alongside each Envoy proxy collaborate with istiod to automate key and certificate rotation:

Istio supports two authentication types: peer authentication and request authentication. Peer authentication secures service-to-service communication with Istio's mutual TLS solution. Request authentication handles end-user authentication through JSON Web Token (JWT) validation using a custom authentication provider or OpenID Connect (OIDC) provider.

Istio enforces service access control by applying authorization policies. These policies regulate inbound traffic in the Envoy proxy, allowing access control at mesh, namespace, and service levels.

6.3. Observability

Istio generates comprehensive telemetry, including metrics, distributed traces, and access logs, for all service communication within the mesh. This telemetry encompasses proxy-level, service-oriented, and control plane metrics.

Previously, Mixer was the central component in Istio's telemetry architecture. However, Telemetry v2 replaces Mixer's features with Envoy proxy plugins:

Istio creates distributed traces through Envoy proxies and supports various tracing backends like Zipkin, Jaeger, Lightstep, and Datadog. Trace generation sampling rate is configurable. Additionally, Istio generates access logs for service traffic in customizable formats.

7. Dive into Istio

Enough background, let's see Istio in action! We'll install Istio on a Kubernetes cluster and use a simple microservices app to showcase its power.

7.1 Installation

Istio installs in various ways, but downloading and extracting the latest release for your OS (like Windows) is easiest. The extracted package includes the istioctl client in the bin folder. Use istioctl to install Istio on your Kubernetes cluster:

istioctl install --set profile=demo -y

kubectl label namespace default istio-injection=enabled

We're using kubectl here, assuming you have a Kubernetes cluster (like Minikube) and the Kubernetes CLI kubectl set up.

7.2 Sample Application

Imagine a simple online ordering app for this demo. It consists of three microservices that work together to fulfill orders:

We won't delve into microservice details, but they're easy to create with Spring Boot and REST APIs. Crucially, create Docker images for these microservices to deploy on Kubernetes.

7.3 Deployment

Deploying containerized workloads on Kubernetes clusters like Minikube is simple. Use Deployment and Service resources to declare and access the workload. Typically, define them in a YAML file:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: order-service
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: order-service
        version: v1
    spec:
      containers:
      - name: order-service
        image: kchandrakant/order-service:v1
        resources:
          requests:
            cpu: 0.1
            memory: 200
---
apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  selector:
    app: order-service

kubectl apply -f booking-service.yaml -f inventory-service.yaml -f shipping-service.yaml

Since we enabled auto-injection of Envoy sidecar proxies for the default namespace, everything's handled. Or, manually inject Envoy sidecar proxies using istioctl's kube-inject command.

7.4 Accessing the Application

Istio primarily handles mesh traffic. By default, traffic to or from outside the mesh is blocked. Istio uses gateways to manage inbound and outbound mesh traffic, giving you precise control over what enters or leaves. Istio offers preconfigured gateway proxy deployments: istio-ingressgateway and istio-egressgateway.

Create a Gateway and Virtual Service for your app:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: booking-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: booking
spec:
  hosts:
  - "*"
  gateways:
  - booking-gateway
  http:
  - match:
    - uri:
        prefix: /api/v1/booking
    route:
    - destination:
        host: booking-service
        port:
          number: 8080

8 Common Istio Use Cases

We've deployed a simple app on Kubernetes with Istio. Let's explore Istio's powerful features and see how they can enhance our application.

8.1 Request Routing

Imagine deploying multiple versions of a microservice like shipping-service. You want to gradually introduce new features without impacting all users. Request routing lets you do this by directing a portion of traffic to the latest version.

Use virtual service routing rules to achieve this.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: shipping-service
spec:
  hosts:
    - shipping-service
  http:
  - route:
    - destination:
        host: shipping-service
        subset: v1
      weight: 90
    - destination:
        host: shipping-service
        subset: v2
      weight: 10
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: shipping-service
spec:
  host: shipping-service
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

Routing rules can also filter traffic based on headers or other attributes. The destination field specifies the target service for matching requests.

8.2 Circuit Breaking

Prevent cascading failures with circuit breakers. This pattern detects errors and temporarily stops traffic to failing services, protecting your application's overall health.

Istio's DestinationRule lets you configure circuit breaking behavior for services like inventory-service.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: inventory-service
spec:
  host: inventory-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 1
      http:
        http1MaxPendingRequests: 1
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutive5xxErrors: 1
      interval: 1s
      baseEjectionTime: 3m
      maxEjectionPercent: 100

By limiting maximum connections, pending requests, and requests per connection, you can effectively manage traffic and prevent overload.

8.3 Enabling Mutual TLS

Mutual TLS ensures secure communication between services by requiring both parties to authenticate. Istio automatically enables mutual TLS for services using its proxies.

While Istio enforces mutual TLS between proxied services, plain-text traffic can still reach services without proxies. Use PeerAuthentication policies to enforce mutual TLS across your entire mesh.

apiVersion: "security.istio.io/v1beta1"
kind: "PeerAuthentication"
metadata:
  name: "default"
  namespace: "istio-system"
spec:
  mtls:
    mode: STRICT

You can apply mutual TLS at the mesh, namespace, or service level. Service-specific policies override namespace-wide settings.

8.4 Access Control With JWT

JSON Web Tokens (JWT) are a standard for securely transmitting user information. They're widely used for authentication and authorization.

Istio's AuthorizationPolicy lets you control access to services like booking-service based on JWT claims.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: require-jwt
  namespace: default
spec:
  selector:
    matchLabels:
      app: booking-service
  action: ALLOW
  rules:
  - from:
    - source:
       requestPrincipals: ["testing@standupcode.com/testing@standupcode.io"]

Require valid JWTs with specific claims for authorized access. Istio creates a requestPrincipal attribute from the JWT's issuer and subject claims.

9. Final Thoughts

Istio simplifies managing common challenges in distributed microservice architectures. However, its complexity can add overhead to deployments. Like any tool, Istio isn't a magic solution and requires careful consideration.

9.1. Is a Service Mesh Always Necessary?

While service meshes offer advantages, consider these drawbacks:

Service meshes offer benefits, but careful evaluation of your application's complexity is crucial. Weigh the advantages against the added complexity.

9.2. Istio Alternatives

Istio is a popular service mesh backed by industry leaders, but it's not the only option. Here's a brief overview of Linkerd and Consul.

Linkerd is a Kubernetes-native, open-source service mesh gaining popularity. It's a CNCF incubating project and works similarly to Istio, using TCP proxies. Linkerd's Rust-based micro-proxy is called Linkerd-proxy.

Linkerd is generally simpler than Istio due to its Kubernetes focus. However, its feature set closely resembles Istio's, and its core architecture is similar. Linkerd consists of a user interface, data plane, and control plane.

Consul is an open-source service mesh from HashiCorp that integrates with other HashiCorp infrastructure tools. Consul's data plane supports both proxy and native integration models. It offers a built-in proxy but can also work with Envoy.

Consul operates beyond Kubernetes, supporting platforms like Nomad. It uses agents on each node for health checks and communicates with Consul servers for data storage and replication. While offering standard service mesh features, Consul is more complex to deploy and manage.

10. Wrapping Up

This tutorial has explored the fundamentals of service mesh architecture and Istio's powerful capabilities. We've delved into Istio's core structure, components, and practical applications. By the end, you should have a solid grasp of how to install and utilize Istio for common scenarios.