Kubernetes Gateway API
Photo by Ani Adigyozalyan

Kubernetes Gateway API

50 min read

In November 2025, Kubernetes announced that NGINX Ingress Controller, one of the most widely adopted ingress controllers that present in more than 40% of Kubernetes clusters, would be deprecated in March 2026.

That news has marked a turning point in the Kubernetes Gateway API adoption, positioning Gateway API as the successor to the Ingress API.

In this post, we will explore Gateway API, how it differs from the Ingress API, and what options you have for migration.

The Ingress Evolution

In the early days of Kubernetes, around 2014, Service resources were the main Kubernetes-native way to expose applications.

At that time, Kubernetes already had roughly the same Service types) we use today:

  • ClusterIP: Gives the Service a stable DNS name inside the cluster, but does not make it accessible from outside the cluster.
  • NodePort: Exposes a specific port on every node in the cluster, allowing external traffic to reach the Service through that port. If the node IPs are publicly accessible, this can expose the Service to the outside world.
  • LoadBalancer: Provisions an external load balancer with a public IP address and forwards traffic to the Service.
  • ExternalName: Provides an in-cluster alias for an external service by adding a CNAME record to the cluster DNS.

Out of these four, only NodePort and LoadBalancer can directly help us expose Services outside the cluster.

NodePort is a fundamental primitive, but it is too low-level for most real-world use cases.

With NodePort, you get a set of node IPs and port numbers that correspond to each Service. There is no built-in external load balancing between nodes, so you would need to implement that yourself. You would also need to handle path-based routing yourself if you wanted a single external endpoint that hides the internal cluster topology.

NodePort Service to Expose Services
NodePort Service to Expose Services

You would need to keep the port mapping in sync between that custom LoadBalancer and actual Kubernetes cluster setup. That’s rather tedious to setup.

The LoadBalancer Service type is more useful in that sense. It builds on top of NodePort by exposing a port on all cluster nodes, but it also provisions an external L4 load balancer, TCP or UDP, that forwards traffic to that NodePort.

The provisioning part happens automatically thanks to the Cloud Controller Manager (or CCM).

Using LoadBalancer Service to Expose Services
Using LoadBalancer Service to Expose Services

That is not great because:

  • You would have a set of public IPs to manage and route between in order to reach specific Services. In practice, you would likely need yet another external proxy to handle the routing.
  • Each external IP and load balancer instance costs money, so maintaining hundreds of them would quickly become expensive.
  • There would be no single central place to manage incoming traffic. That matters because ingress endpoints are usually the first place attackers probe when trying to get into a system.

If we think about this problem a bit more, we can see that we do not need a separate external load balancer for every Service. Instead, we could have a single LoadBalancer Service that accepts all incoming traffic and forwards it to a dedicated reverse proxy Deployment, such as NGINX. That reverse proxy would then route traffic to the right backend Service based on the request path, hostname, or other request properties.

Using LoadBalancer Service to Expose Services with a Reverse Proxy
Using LoadBalancer Service to Expose Services with a Reverse Proxy

This setup is what eventually gave birth to the Ingress API and Ingress Controllers.

Ingress Controller

In 2015, the Kubernetes team officially introduced the Ingress API as a way to define rules for routing external HTTP(S) traffic to Services inside the cluster.

The Ingress API consists of two main resources: IngressClass and Ingress.

Kubernetes Ingress API
Kubernetes Ingress API

It defines a common interface, but it doesn’t come with a specific implementation out of the box. To make these rules actually work, you need to install an Ingress Controller that knows how to interpret and apply them.

IngressClass

IngressClass is a cluster-wide resource that tells Kubernetes which controller should handle a given set of Ingress resources, and which parameters that controller should use.

IngressClass makes it possible to run multiple Ingress Controllers side by side in the same cluster. Each controller watches its own set of Ingress resources, selected by the corresponding IngressClass.

Multiple Ingress Controllers in the same cluster
Multiple Ingress Controllers in the same cluster
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: alb
spec:
  controller: ingress.k8s.aws/alb
  parameters:
    scope: Cluster
    apiGroup: elbv2.k8s.aws
    kind: IngressClassParams
    name: alb-ingress-params
---
apiVersion: elbv2.k8s.aws/v1beta1
kind: IngressClassParams
metadata:
  name: alb-ingress-params
spec:
  scheme: internal
  ipAddressType: dualstack
  tags:
      - key: costcenter
        value: product-eng
  loadBalancerAttributes:
      - key: deletion_protection.enabled
        value: "true"
      - key: idle_timeout.timeout_seconds
        value: "120"

IngressClass parameters are an optional way to pass additional, implementation-specific configuration to the Ingress Controller.

Ingress Controllers also need the right ClusterRole permissions to read and use IngressClass resources.

Ingress Resources

Ingress is the main resource in the Ingress API. It combines a few things:

  • It defines rules for matching incoming requests and routing them to internal Services using path-based matches, such as exact or prefix matches, and host-based policies.
  • It defines TLS configuration for incoming traffic.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  namespace: application
spec:
  ingressClassName: alb # Reference to the IngressClass that should handle this Ingress resource
  tls:
  - hosts: # Handle HTTPS traffic for these hosts
      - app.romaglushko.com
    secretName: ingress-tls-secret # contains a TLS private key and certificate
  rules:
  - host: app.romaglushko.com # Host-based routing
    http:
      paths: # Path-based routing
      - path: /users
        pathType: Prefix
        backend:
          service:
            name: users # Internal microservice Service name
            port:
              number: 8000
      - path: /orders
        pathType: Prefix
        backend:
          service:
            name: orders # Internal microservice Service name
            port:
              number: 8000

Once the Ingress resource is created, the Ingress Controller picks it up and updates its own configuration to apply the new routing rule. That’s it.

The Problem

So what’s the problem with the Ingress API? It seems like a simple solution to exposing Services to the outside world, right?

Well, yes, until you start using it in real life setups. That’s where a few problems show up:

Very Limited API Surface

The problem is that the Ingress API is simplistic rather than simple. It covers only the bare minimum of what we usually want to configure for ingress traffic. Just think about the features you might want from something like NGINX:

  • request timeouts
  • CORS
  • max body size limit
  • sticky cookie sessions
  • canary traffic splits
  • rate limiting
  • header-, query-, cookie-based routing
  • header modifications

The Ingress resource supports none of these directly. Because of that, custom annotations became the canonical way to pass extra configuration to Ingress Controllers. Some controllers, like Traefik, use their own CRDs instead.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  namespace: application
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, OPTIONS"
spec:
  ingressClassName: nginx
  rules:
  - host: app.romaglushko.com # Host-based routing
    http:
      paths: # Path-based routing
      - path: /static
        pathType: Prefix
        backend:
          service:
            name: webstatic
            port:
              number: 3000

What if your Ingress resources need to work with multiple Ingress Controllers at the same time? Because there is no unified way to configure these common features, each Ingress Controller usually has its own set of annotations.

The more features you enable, the less portable your Ingress resources become.

On top of that, Ingress is only concerned with HTTP(S) routing. Support for other protocols, such as gRPC (L7), TCP, or UDP (L4), depends entirely on the specific implementation and is usually configured through custom annotations.

Rigid Extensibility

Custom annotations are a pretty hacky way to extend the Ingress API. They are just string-based key-value pairs. But often, you need to pass more complex configuration, so you end up serializing it into a string and putting it into an annotation.

It works, but it is not very expressive.

No Policy Enforcement

When application teams create Ingress resources, they can add basically any annotations to them, including annotations that change the behavior of the Ingress Controller. For example, they might disable rate limiting even though the platform team expects it to be enabled for all routes. There is no built-in mechanism in the Ingress API to enforce these global behaviors.

The best you can do is use an external policy engine like Kyverno or OPA Gatekeeper.

Blurry Ownership Boundary

Ingress resources mix several types of configuration:

  • Routing rules, usually owned by application teams because they know how their API should behave.
  • Hostname and port configuration, usually owned by platform teams because they manage DNS, load balancers, and VPC/networking setup.
  • TLS configuration, usually owned by platform teams because they handle certificate provisioning.
  • Custom annotations, which may enable functionality owned by either application or platform teams.

So where should Ingress resources live?

Imagine a complex system deployed as an umbrella Helm chart with multiple microservice subcharts. In that setup, Ingress resources usually live in the application subcharts, close to the Services they expose.

That makes sense for routing rules. But the platform team may still need to update parts of the same Ingress resources from time to time. They may also need to enforce how some parts of the Ingress configuration are rendered.

If we apply the single-responsibility principle here, Ingress clearly has more than one reason to change.

That is a good signal that it carries too much responsibility and should probably be split into more focused resources.

No Secure Cross-Namespace Support

Ingress resources cannot reference Services or TLS secrets from namespaces other than their own.

For example, rules[].backend.service does not even have a field for specifying a namespace. As a result, the most natural way to define an Ingress resource is to place it in the same namespace as the microservice resources it exposes.

There is a workaround though for targeting Services in namespaces other than the one where the Ingress resource was created.

It involves creating an ExternalName Service in the same namespace as the Ingress resource. This ExternalName Service then points to the target Service by its in-cluster DNS name.

Ingress ExternalName Bridge
Ingress ExternalName Bridge

With this workaround, an Ingress resource can indirectly target Services across namespaces. There is one tiny problem, however, this is exactly what we call the confused deputy attack.

Gateway API

Kubernetes Gateway API Logo
Kubernetes Gateway API Logo

You can think of Gateway API as the next generation of the Ingress API. It tries to address the limitations of its predecessor by:

Kubernetes Gateway API
Kubernetes Gateway API

Let’s dive deeper into the Gateway API resource model.

GatewayClass

Similarly to IngressClass, GatewayClass defines a set of Gateways owned by a specific Gateway controller deployment.

The GatewayClass resource has effectively the same meaning and structure as IngressClass:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: gateway-public
  namespace: envoy-gateway
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: config.gateway.envoyproxy.io
    kind: EnvoyProxy
    name: gateway-public-config
    namespace: envoy-gateway
---
# A custom resource that configures how to deploy Gateway proxy
apiVersion: config.gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: gateway-public-config
  namespace: envoy-gateway
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        replicas: 3

GatewayClass can also reference a custom resource that contains additional implementation-specific Gateway configuration. This may include:

  • how a specific Gateway proxy should be exposed
  • default implementation-specific settings for bootstrapping and running the Gateway proxy
  • how the Gateway proxy deployment should be rolled out and scaled
  • other controller-specific options

Gateway

The Gateway resource represents a dynamically provisioned edge gateway. It abstracts the infrastructure that accepts incoming traffic and routes it to the right backend services.

In the Ingress design, the Ingress Controller effectively played this role, so we can roughly think of it as a statically provisioned Gateway instance.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: gateway-public
  namespace: envoy-gateway
spec:
  gatewayClassName: gateway-public
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      hostname: "*.romanglushko.com"
      allowedRoutes:
        namespaces:
          from: Selector
            selector:
              matchLabels:
                app: "web"
    - name: https
      protocol: HTTPS
      port: 443
      hostname: "*.romanglushko.com"
      tls:
        mode: Terminate
        certificateRefs:
          - name: app-tls
      allowedRoutes:
        namespaces:
          from: Selector
            selector:
              matchLabels:
                app: "web"
    - name: tls-passthrough-db
      protocol: TLS
      port: 8443
      hostname: "db.romanglushko.com"
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: Selector
            selector:
              matchLabels:
                app: "db"
        kinds:
          - kind: TLSRoute

Each Gateway references a GatewayClass, which tells Kubernetes which Gateway controller should provision and manage it.

Beyond that, the most important part of a Gateway configuration is its list of listeners.

Listeners

Gateway listeners define how a Gateway accepts incoming traffic.

Each listener represents a distinct entry point, usually described by a combination of port, protocol, and hostname. Listeners can also configure TLS termination. In the Ingress world, this information lived directly inside Ingress resources.

Listeners are the most fine-grained level at which Routes can attach to a Gateway.

ListenerSet

Gateway listeners are a great way to centralize traffic entry point settings, but they do not necessarily work well for use cases where you need hundreds of them.

The primary use case is HTTPS listeners with per-service hostnames and TLS settings, instead of a single wildcard TLS certificate. In this case, you may end up with as many listeners as there are microservices in the system.

If you tried to model this directly with a single Gateway, you would quickly run into two problems:

  • Gateway supports only up to 64 listeners
  • Gateway becomes a coordination bottleneck when many teams actively manage listeners and their TLS settings

Therefore, we need a way to decentralize listener configuration and make it more multi-tenant.

That is where the ListenerSet resource comes in handy.

apiVersion: gateway.networking.k8s.io/v1
kind: ListenerSet
metadata:
  name: demp-app
  namespace: demo-app
spec:
  parentRef:
    kind: Gateway
    group: gateway.networking.k8s.io
    name: gateway-public
    namespace: envoy-gateway
  listeners:
  - name: demo
    hostname: demo.romaglushko.com
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        group: ""
        name: demo-app-cert

The Allowed Listeners & Routes

After Gateway API separated the concept of Gateways from Routes, it created a new problem: how do you control which tenants can attach Routes to which listeners?

Gateway listeners are shared infrastructure, and different listeners may serve very different purposes. For example:

  • in our example above, it does not make sense to attach HTTPRoutes to the tls-passthrough-db listener because it is just a passthrough channel to the database deployment
  • it also does not make sense to attach Routes to that listener from namespaces other than db

Since Routes are intended to be added in a self-managed way, Gateway API needs another control mechanism to prevent misconfiguration. That mechanism is called allowedRoutes.

# ...
- name: tls-passthrough-db
  protocol: TLS
  port: 8443
  hostname: "db.romanglushko.com"
  tls:
    mode: Passthrough
  allowedRoutes:
    namespaces:
      from: Selector
        selector:
          matchLabels:
            app: "db"
    kinds:
      - kind: TLSRoute

allowedRoutes helps establish trust between Gateways or ListenerSets and Routes from a specific set of namespaces or of specific route types.

Routes

Gateway listeners can accept incoming traffic, but they do not know what to do with it next. This is where Route resources come in.

Routes are the meat of the Gateway API. This is where most of its flexibility really lives. There are five Route resources that cover several popular protocols:

  • HTTPRoute - greatly enhanced and extended HTTP traffic routing
  • GRPCRoute - gRPC-aware routing
  • TLSRoute - TLS SNI-based routing
  • TCPRoute and UDPRoute - forwarding TCP/UDP traffic from the Listener port directly to the backend service

All Route resources, sometimes collectively called xRoutes, have a similar envelope structure:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: auth
  namespace: auth
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: gateway-public
    namespace: envoy-gateway
    sectionName: https
  hostnames:
  - "auth.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: auth-service
      port: 8080

The parentRefs section is a typed object reference to a parent resource that can accept Routes. In most cases, this is a Gateway, but it can also be a ListenerSet, a Service in the case of service mesh, or another custom resource supported by a Gateway API implementation.

Compared to the typed object references we have seen in IngressClass and GatewayClass parameters, parentRefs are more precise. They can include optional sectionName and port (in case of UDP/TCP routes) fields that allow a Route to target a specific listener, or a specific set of listeners, on the parent resource.

The rules section contains the actual protocol-specific routing rules, filters, and other configuration. Each rule contains:

  • a list of matches that select the requests to which the rule should apply
  • an optional list of filters that can process the request or response
  • an optional list of backendRefs used for forwarding traffic and doing weighted load balancing. The backendRefs are optional because filters may completely handle the request without forwarding it further upstream.

If the protocol allows it, a Route resource can also include a hostnames field. This enables host-based routing at the Route level, rather than only at the listener level.

To walk you through Routes configuration, I will group related features together and show an example of solving a common or interesting use case with each group. You can find more configuration examples in the official Gateway API guides.

HTTPRoute

HTTPRoute lets you define routing rules for HTTP(S) traffic at the L7 level.

Traffic Matching

In terms of traffic matching, HTTPRoute supports:

  • Ingress-like path- and host-based routing
  • header-, query-, and method-based matching rules

For example, if you try to roll out a testing, canary release that should to be available to the internal clients only, you can leverage header-based matching to route traffic to the testing deployment of your microservice:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: payment
  namespace: payment
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: gateway-public
      namespace: envoy-gateway
      sectionName: https
  rules:
    # This is a canary rule
    - matches:
        - path:
            type: PathPrefix
            value: /payment
          headers:
            - name: X-Release-Track
              value: canary
      backendRefs:
        - name: payment-service-canary
          port: 8080
      timeouts:
        # not willing to wait more than 500ms
        # for the response from the canary deployment
        request: 300ms
 
    # This is a rule for regular traffic
    - matches:
        - path:
            type: PathPrefix
            value: /payment
      backendRefs:
        - name: payment-service
          port: 8080

The Gateway data plane should apply the most specific matching rule to incoming traffic, so the order of rule definitions should not matter.

URL Rewrites

HTTPRoute can also rewrite parts of the incoming request URL before the request reaches the backend Service by using filters:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: shipping
  namespace: shipping
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: gateway-public
      namespace: envoy-gateway
      sectionName: https
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /api/v1/shipping
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplacePrefixMatch
          replacePrefixMatch: /shipping
    backendRefs:
    - name: shipping-service
      port: 8000

Header Modifications

If you need to add, remove, or modify request or response headers, you can use another filter type called HeaderModifier:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: docs
spec:
  parentRefs:
  - name: gateway-public
    namespace: envoy-gateway
  hostnames:
  - "docs.romaglushko.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    filters:
    - type: ResponseHeaderModifier
      responseHeaderModifier:
        add:
        - name: X-API-Gateway
          value: "true"
    backendRefs:
    - name: docs
      port: 8000

HTTPRoute also has a dedicated CORS filter for configuring Cross-Origin Resource Sharing. Conceptually, this is a special case of response header modification, but Gateway API exposes it as a separate filter type:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: assets
spec:
  parentRefs:
  - name: gateway-public
    namespace: envoy-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /assets
    backendRefs:
    - name: docs
      port: 8080
    filters:
    - cors:
        allowOrigins:
        - "https://www.romaglushko.com"
        - "https://*.romaglushko.com"
        allowMethods:
        - GET
        - OPTIONS
        allowHeaders:
        - "*"
        exposeHeaders:
        - "x-header-3"
        - "x-header-4"
        allowCredentials: true
        maxAge: 3600
      type: CORS

Redirects

Gateway API can return a redirect response to the client instead of forwarding the request to the backend. This supports regular path-based redirects with 3xx status codes, as well as HTTP-to-HTTPS redirects.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: docs
spec:
  parentRefs:
  - name: gateway-public
    namespace: envoy-gateway
  hostnames:
  - api.romaglushko.com
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /old-page
    filters:
    - type: RequestRedirect
      requestRedirect:
        path:
          type: ReplaceFullPath
          replaceFullPath: /old-page
        statusCode: 301

Traffic Control

Gateway API also covers several common traffic control use cases.

For example, you can split traffic between different backend services using weights:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: payments
  namespace: payments
spec:
  parentRefs:
   - name: gateway-public
     namespace: envoy-gateway
  rules:
  - backendRefs:
    - name: payments-blue
      port: 8080
      weight: 90
    - name: payments-green
      port: 8080
      weight: 10

This is helpful for implementing blue-green deployments, A/B testing, and similar rollout patterns.

Another useful traffic control pattern is traffic mirroring.

Traffic mirroring lets you send a copy of production traffic to an additional backend service, while the original request still goes to the primary backend. In Gateway API, this is configured with the RequestMirror filter:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: comments
spec:
  parentRefs:
  - name: gateway-public
    namespace: envoy-gateway
  rules:
  - backendRefs:
    - name: comments
      port: 8080
    filters:
    - type: RequestMirror
      requestMirror:
        backendRef:
          name: comments-v2
          port: 8080

Traffic mirroring is one of the key building blocks of tap-and-compare testing, where you send production traffic to both the old and new versions of your microservice, for example during a major refactoring or migration, and compare the results that are expected to stay the same.

Sticky Sessions

Gateway API also supports sticky sessions, also known as session persistence.

You can configure a cookie or header as a session marker, so requests from the same client are consistently routed to the same backend instance.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
# ...
spec:
  parentRefs:
  # ...
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: notifications
          port: 8080
      sessionPersistence:
        sessionName: ws_sess
        type: Cookie
        absoluteTimeout: 24h

External Authentication

HTTPRoute also supports an experimental external authentication filter based on either gRPC or HTTP.

With this filter, the Gateway sends request headers to an external authentication service before forwarding the request to the backend. The request body can also be forwarded, but that must be enabled explicitly.

In the gRPC case, the external authentication service must implement Envoy’s ext_authz protobuf API.

In the HTTP case, the external authentication service must return a 200 response for successful authentication. Any other response status is treated as an authentication failure.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
# ...
spec:
  parentRefs:
  # ...
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /account/me
      filters:
        - type: ExternalAuth
          externalAuth:
            protocol: HTTP
            backendRef:
              # reference of the auth service
              group: ""
              kind: Service
              name: auth
              port: 8000
            http:
              path: /auth
              allowedHeaders:
                # the headers to be forward to the auth service
                - authorization
                - cookie
                - x-api-key
              allowedResponseHeaders:
                # the headers to copied from
                # the auth service response to upstream backend
                - x-user-id
                - x-org-id
      backendRefs:
        - name: users
          port: 8080

Retries & Timeouts

HTTPRoute can also configure retries for a given route:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
# ...
spec:
  parentRefs:
  # ...
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api
      backendRefs:
        - name: permadeletion
          port: 8000
      retry:
        codes:
          - 500
          - 502
          - 503
          - 504
        attempts: 3
        backoff: 100ms

BackendTrafficPolicy additionally provides a way to configure a global retry budget, which helps avoid retry storms.

GRPCRoute

Technically speaking, HTTPRoute could handle gRPC traffic because gRPC is based on HTTP/2.

However, there are a few good reasons to model gRPC with its own resource:

  • Not all filters and configuration options that make sense for HTTPRoute also make sense for gRPC. For example, URL rewrites are not a common problem in the gRPC world. Separating HTTPRoute and GRPCRoute allows each resource to evolve independently in a way that better fits its protocol.
  • gRPC is based on HTTP/2, but it still has its own protocol-specific details that an API Gateway should understand out of the box. For example, a gRPC response may return 200 as the HTTP status code while still representing an application-level error through grpc-status and grpc-message response headers. Understanding this distinction is important for proper retries and accurate request metrics.
  • Finally, it is a better user experience when you can define routing rules in high-level gRPC terms.

That is why Gateway API has a dedicated route type called GRPCRoute.

Here is an example of a GRPCRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
  name: users
spec:
  parentRefs:
  # ...
  hostnames:
  - "grpc.romaglushko.com"
  rules:
  - matches:
    - method:
        service: com.romaglushko.User
        # method: Login - you can be more specific by matching gRPC method, too
    backendRefs:
    - name: users
      port: 50051

The HTTPRoute path matcher is replaced by the method matcher in GRPCRoute. It still matches the request path under the hood, but exposes it in gRPC terms: service and method.

Just like HTTPRoute, GRPCRoute can work with request and response headers, but it does not decode the gRPC payload. This means you cannot route based on a specific protobuf field. For example, you cannot express logic like this:

  • if this is a Login command and LoginRequest.type == "mobile", route the request to the users-mobile service

GRPCRoute supports a subset of HTTPRoute filters, including:

TCPRoute and UDPRoute

So far, we have reviewed two high-level application-layer route types. Now let’s look at two lower-level transport-layer routing capabilities in Gateway API: TCPRoute and UDPRoute.

TCPRoute and UDPRoute take traffic that lands on the listener port they are attached to and simply forward it to a backend service.

That is pretty much all they can do right now. They do not support matchers or filters.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: gateway-dbs
  namespace: dbs
spec:
  gatewayClassName: envoy-gateway
  listeners:
    - name: redis
      protocol: TCP
      port: 6379
      allowedRoutes:
        kinds:
          - kind: TCPRoute
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
  name: redis
  namespace: dbs
spec:
  parentRefs:
    - name: gateway-dbs
      namespace: dbs
      sectionName: redis
  rules:
    - backendRefs:
        - name: redis
          port: 6379

Both route types support weighted load balancing across multiple backend services.

TLSRoute

Finally, there is another special-purpose route type called TLSRoute.

TLSRoute routes TLS-encrypted traffic to backend services based on Server Name Indication (SNI), which is provided during the TLS handshake.

This is commonly used for TLS passthrough: the Gateway inspects the SNI value, chooses the backend, and passes the encrypted connection through without terminating TLS. In this mode, the backend service is responsible for terminating TLS.

Some implementations like Envoy Gateway or [kgateway] may also support TLS termination at edge for TLSRoute, but that is an extended capability rather than the baseline behavior.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
# ...
spec:
  # ...
  listeners:
    # ...
    - name: tls-passthrough-db
      protocol: TLS
      port: 8443
      hostname: "db.romanglushko.com"
      tls:
        mode: Passthrough
---
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
  name: dbs
spec:
  parentRefs:
  - name: gateway-public
    sectionName: tls-passthrough-db
  hostnames:
  - "db.romanglushko.com" # a required configuration
  rules:
  - backendRefs:
    - name: postgres-ha-cluster
      port: 8443

TLSRoute requires hostnames to be specified on the Route. These hostnames are matched against the SNI value from the TLS handshake, and they must intersect with the hostname configured on the Gateway listener.

Other than that, TLSRoute doesn’t support matchers or filters. It does, however, support weighted load balancing across target backend services.

Route Filter Extensions

HTTPRoute and GRPCRoute include an extension mechanism for custom filters and request/response processing through the extensionRef filter. A Gateway API implementation may provide additional implementation-specific filters that extend what you can configure with Routes.

This is possible because extensionRef uses a typed object reference to point to another Kubernetes resource.

For example, Envoy Gateway provides an HTTPRouteFilter CRD that can return a direct response without requiring any backend service:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: maintenance
spec:
  parentRefs:
  # ...
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /maintenance
      filters:
        - type: ExtensionRef
          extensionRef:
            group: gateway.envoyproxy.io
            kind: HTTPRouteFilter
            name: maintenance-response
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: HTTPRouteFilter
metadata:
  name: maintenance-response
  namespace: default
spec:
  directResponse:
    contentType: text/plain
    statusCode: 503
    body:
      type: Inline
      inline: "Service is temporarily unavailable."

Backend Targets

By default, Gateway API supports the following resources as backend targets:

However, backend targets are also specified through a typed object reference. This means the set of supported backend targets can be extended with other custom resources supported by your Gateway API implementation.

For example, Envoy Gateway additionally supports a custom Backend resource that helps decouple upstream configuration from in-cluster microservices. With this resource, you can target external upstreams by FQDN, IP address, or even resources local to the data plane pod, such as sidecars exposed through UNIX sockets:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: external-httpbin
spec:
  parentRefs:
  # ...
  hostnames:
    - "wiki.romaglushko.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - group: gateway.envoyproxy.io
          kind: Backend
          name: httpbin
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
  name: httpbin
spec:
  endpoints:
    - fqdn:
        hostname: grokipedia.com
        port: 80

ReferenceGrant

Gateway API is very explicit about cross-namespace references. It makes them a first-class concept in the standard design. However, cross-namespace references require care because they may introduce security risks.

Let’s review the following situation, where an attacker manages to compromise one namespace in a cluster.

The Confused Deputy Attack in Action
The Confused Deputy Attack in Action

In the compromised namespace, the attacker has permissions to create new Ingress, Service, and EndpointSlice resources.

This is enough to perform a confused deputy attack that leaks access to otherwise protected services:

  • The attacker creates a new Ingress resource that points to a new Service in the compromised namespace.
  • The created Service is selectorless, so it does not match any deployment. This is a valid Kubernetes configuration, and it allows the attacker to provide the corresponding EndpointSlice manually.
  • The attacker then creates an EndpointSlice for that Service and forges the endpoint pool to include IPs of protected backend pods from another namespace.

Now the attacker can align the ports between the Ingress, Service, and EndpointSlice, effectively creating a second front door to the protected API.

A shorter way to reach the same result would be to use an ExternalName Service, just like we did in the Ingress section.

Hopefully, this illustrates why cross-namespace references need to be handled carefully.

To avoid making this problem even worse in Gateway API, which intentionally allows resources like Secrets and Services to be referenced across namespaces, the Kubernetes team introduced the ReferenceGrant resource.

ReferenceGrant is another bidirectional trust mechanism. It allows a namespace to define which other namespaces are allowed to reference its resources.

A Gateway Controller considers ReferenceGrant resources whenever it encounters cross-namespace references to backend targets or TLS secrets.

ReferenceGrants
ReferenceGrants

ReferenceGrant adds a layer of protection that makes confused deputy attacks harder to perform. Without it, cross-namespace references would be as simple as pointing a Route to an arbitrary Service in another namespace.

How ReferenceGrant Helps Mitigate a Confused Deputy Attack
How ReferenceGrant Helps Mitigate a Confused Deputy Attack

That said, ReferenceGrant only makes a cross-namespace reference legitimate. It is not concerned with what happens after traffic is sent to the referenced backend. That backend may still resolve to, proxy to, or otherwise forward traffic to an unauthorized destination.

NetworkPolicies can certainly help here, as long as they are precise enough to block access to the protected API port from unauthorized sources.

Policies

Policy attachment is a powerful extension pattern that helps to apply a new behaviour or effect (also knows as Policy) to one or multiple Gateway API components hierarchically without modifying their original Kubernetes resources. That way we can augment Gateway API with a custom, implementation-specific functionality.

Kubernetes Gateway API Policies
Kubernetes Gateway API Policies

Authentication is a very good example of a Policy in action:

  • you may want to apply authentication globally to the whole Gateway (or ListenerSet) resource meaning that it will hierarchically affect all Routes that are attached to it right now and in the future. At the same time, you may want to have Route level exceptions like public routes.
  • authentication may be controlled by the team that has nothing to do with Gateway or specific Routes resources, so they are not supposed to modify those resources directly.
  • Even though there are well-established standards in the authentication space (e.g. OAuth 2, OIDC, etc.), the authentication configuration still highly depends on the Gateway API implementation. It may be challenging to find practically useful abstraction that works for all implementations (unless it’s completely delegated as in HTTPRoute External Authentication).

These are the considerations that Policy attachment was designed around. In most cases, Policies are just custom resources that are attached to the needed scopes and contains the implementation-specific configuration.

For example, Envoy Gateway comes with SecurityPolicy resource definition that helps to configure JWT token validation like this:

apiVersion: gateway.envoyproxy.io/v1alpha1 # a custom resource
kind: SecurityPolicy
metadata:
  name: auth
  namespace: envoy-gateway
spec:
  targetRefs: # the target section: typed object references that represent the scope the policy is applied to
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: gateway-public
  jwt: # the intent section: a custom, implementation specific configuration
    providers:
      - name: auth0
        issuer: "https://rg.eu.auth0.com/"
        audiences:
          - "https://api.romaglushko.com"
        remoteJWKS:
          uri: "https://rg.eu.auth0.com/.well-known/jwks.json"

Since Policies convey custom configurations, they are a modern, expressive replacement for the custom annotations as configuration pattern that was popular in the Ingress Controller era.

Out of the box, Gateway API includes just two Policies:

  • BackendTLSPolicy that instruct Gateway to connect to the upstream backend via TLS.
  • BackendTrafficPolicy that helps to specify retry budget for given backend. The budget defines the global quote of allowed retries before we just give up and return 503. It acts in a similar manner to circuit breaker. In general, this is a great way to avoid retry storms.

Ownership

There are typically two teams that may own Gateway API resources in your cluster:

  • the Platform team. The Platform team defines GatewayClass, Gateway, and ListenerSet resources, as well as everything needed to expose them, such as LoadBalancers and TLS configuration. The Platform team may also manage a set of global Policies that apply to some or all Gateways.
  • the Application or Service teams. Application teams mostly focus on their own Route resources. They may also apply their own Route-level Policies when needed.
Gateway API Ownership
Gateway API Ownership

Gateway API gives you a lot of flexibility here, so you can come up with many alternative ownership models. For example, Routes may also be owned by the Platform team and kept together in one namespace.

Typical Architecture

Gateway API does not make too many assumptions about the actual implementation architecture, but a common way to implement it is through control plane and data plane separation.

Common Architecture in Gateway API
Common Architecture in Gateway API

The Gateway Controller is a Kubernetes operator that acts as the control plane. It watches Gateway API resources and related CRDs, then builds a combined view of the desired Gateway data plane configuration. To do that, the Controller usually needs extended permissions to read and modify Kubernetes resources.

The Gateway data plane is the proxy that serves traffic according to the configuration produced by the Gateway Controller. Common data planes include Envoy Proxy, NGINX, and custom proxies like Traefik. Gateway Controllers often provision or configure proxy instances based on Gateway resources, but the exact model depends on the implementation. Some implementations may provision one proxy per Gateway for stronger isolation, while others may share a data plane across multiple Gateways.

The control/data plane separation is a good design choice because it helps avoid some of the fundamental security flaws found in the NGINX Ingress Controller.

Ingress Migration

Now that we have gone through the evolution of Ingress, you may be wondering what to do about the deprecation news if you actively use the NGINX Ingress Controller. There are basically four options:

Let’s go through them from the least intrusive option to the most intrusive one.

Do Nothing

Not the best response, but it is technically on the table in some cases.

If you run the NGINX Ingress Controller in a homelab, there may not be much motivation to do anything about it. It works, so you might just leave it alone.

In enterprise setups, though, this is much harder to tolerate. If you operate under legal, security, or compliance frameworks like these, you are usually expected to run maintained and patched software:

  • ISO 27001: Expects vulnerability management, patching, and EOL tracking.
  • SOC 2 Type II: Expects vulnerability scanning, patch management, and remediation SLAs for software in use.
  • HIPAA Security Rule: Legacy, obsolete, or unpatched software needs to be covered by the HIPAA risk analysis.
  • PCI DSS v4.0.1 (Payment Processors): Requires organizations to identify unsupported software components and have a remediation plan. It also defines strict timelines for addressing known critical vulnerabilities.
  • FedRAMP (Government): Expects unsupported software to be replaced with maintained alternatives, such as another product or a supported fork. It also has strict remediation timelines for vulnerabilities of different severities.

For most of these frameworks, EOL software becomes a real problem once a new vulnerability is publicly disclosed.

But how risky is it in practice?

CVE Analysis

Let’s take a look at the CVEs disclosed in the NGINX Ingress Controller over the last five years:

  • 20 vulnerabilities overall
  • 4 High-severity CVEs in 2021, all related to secret leakage
  • 1 High-severity CVE in 2022 related to controller credential leakage
  • 3 High-severity CVEs in 2023-2024
  • 6 CVEs in 2025, including 1 Critical CVE, known as IngressNightmare, and 4 High-severity CVEs related to NGINX configuration injection
  • 6 CVEs in 2026, including 3 High-severity CVEs related to NGINX configuration injection

The 2021–2022 CVEs were mostly about unsanitized, user-provided NGINX configuration in annotations. That opened the door to configuration injection, information disclosure, and secret leakage. Kubernetes added validation to reduce that risk.

The 2023–2024 CVEs were mostly about bypassing those annotation validations.

Then IngressNightmare (CVE-2025-1974) shook things up badly. Before that, you usually needed permission to create or modify Ingress resources to exploit this class of issues. IngressNightmare made the situation worse: any workload with network access to the admission controller could potentially exploit a configuration-injection-like bug and achieve remote code execution in the context of the ingress-nginx controller. From there, the attacker could potentially leak any Secrets the Controller was allowed to access.

The 2026 batch continued the same pattern: more configuration injection issues, this time through annotations, paths, or comments.

These vulnerabilities are not just random security bugs. They keep hitting the same weak spots in the design:

  • The Controller is extremely flexible and exposes a wide range of NGINX functionality. That makes perfect validation very hard, so configuration injection remains a recurring attack vector.
  • The Controller runs the control plane and data plane in the same pod and shares the same filesystem. That increases the blast radius when something goes wrong.
  • The Controller often has access to Secrets across the whole cluster. In that setup, a successful configuration injection can turn into cluster-wide secret leakage.

Given these structural problems, it is reasonable to expect more CVEs in the future. Just staying on the original NGINX Ingress Controller without a migration plan is a risky bet.

Use an NGINX Controller Fork

The simplest path is to switch to a maintained fork of the NGINX Ingress Controller:

  • Chainguard’s fork, although they don’t seem to provide built images because that is part of what they sell.

This reduces the risk of being exposed to newly disclosed CVEs and keeps your system running without major changes or a full migration.

But this is still a short-term solution.

Chainguard is not trying to keep developing the Controller as a long-term project. They are providing best-effort CVE remediation to buy NGINX Ingress Controller users more time to migrate.

Use Another Ingress Controller

Ingress resources are not deprecated in general. Switching to another maintained Ingress Controller is still a very viable option.

Ingress resources are not deprecated in general. Switching to another, maintained Ingress Controller is a very viable option.

The NGINX Ingress Controller is not the only option. There are plenty of fish in the sea, in fact, too many to list here. The most popular options I’ve personally seen in the wild are:

This still requires migrating your annotations across the system. That may be super easy or fairly painful, depending on how deeply you rely on NGINX-specific functionality.

Over time, though, we will likely see less focus on Ingress across the ecosystem as more projects gradually move toward Gateway API. That said, Ingress is still more than fine for the time being.

Migrate to Gateway API

Finally, the only real long-term option is to move from the Ingress API to Gateway API.

This means:

To help you with the last one, the Kubernetes team created ingress2gateway, a useful CLI that does a best-effort automatic conversion from Ingress resources to Gateway API resources. You will still need to double-check custom annotations afterward, because this is usually the trickiest part of an Ingress migration.

You may still need to double-check custom annotations afterward as this is the trickiest part of the Ingress migration. In general case, you would like to make sure that various timeouts and file upload limits are properly migrated as they can silently break your application assumptions if missed or mismatch.

In most cases, you want to make sure that timeouts, file upload limits, body size limits, and similar settings are migrated correctly. If you miss them, or map them incorrectly, you may silently break application assumptions.

You also need to carefully plan the traffic switch from the Ingress Controller’s LoadBalancer to the new Gateway proxy’s LoadBalancer. This does not have to be a big-bang migration.

You can temporarily keep the Ingress Controller as the public entry point and route selected traffic to the Gateway API data plane until you are ready to make Gateway API the main edge gateway.

This lets you expose parts of the system through Gateway API earlier, test it with real traffic, and avoid switching everything at once.

Which Gateway To Use?

Once you decide to move forward with the Gateway API migration, the first big question is which Gateway API implementation to use.

In this last section, I will share a few personal recommendations for picking a Gateway API implementation for a generic API Gateway use case. Of course, it is hard to give universal advice without knowing the full context. So treat this as one more perspective for your due diligence, not as the final answer.

For this article, I define a generic API Gateway use case as:

An extensible gateway for public North-South traffic, deployed in an environment you fully control, with common authentication patterns and flexible rate limiting available out of the box.

These days, an API Gateway is not the only type of gateway you might need. There are also LLM Gateways and Agent Gateways, but I will keep this section focused on the classic API Gateway use case.

Gateway API Conformance

The Kubernetes team has created a standard set of conformance tests to verify that implementations handle the core parts of the Gateway API protocol correctly.

These tests mostly focus on functional behavior. They do not cover many practical non-functional concerns, such as reliability, performance, scalability, or operational complexity.

You should prefer a conformant Gateway API implementation.

If the conformance results for the implementation you are considering are not listed on the official Gateway API website, contact the maintainers and ask for their conformance report.

Public Benchmarks

Besides the official conformance test suite, there are also public benchmarks that focus on things outside the scope of conformance tests, such as reliability and performance.

John Howard, an Istio contributor and Solo.io employee, created a benchmark of the main Gateway API implementations. Solo.io is the company behind kgateway.

John covered useful test cases around Route attachment, propagation, scale, and general traffic-handling performance.

The test suite is based on his own experience optimizing gateways he worked on, so it may be more focused toward certain specific use cases he have seen. That said, I don’t see anything wrong with the test cases themselves.

Use it as another useful perspective during your evaluation.

OSS vs Proprietary

These days, open source has a lot of strong, production-ready Gateway API implementations. So take OSS seriously during your evaluation.

For many organizations, open source should be the default starting point.

Part of the reason is that many features that used to justify buying a proprietary gateway are now becoming commodities. For example, you probably do not want to buy a proprietary implementation just because it supports OAuth2 or OIDC. Many OSS gateway controllers support that out of the box now.

From my experience, open source solutions are often just as good as proprietary ones, and sometimes better.

Plus, with OSS, you can do your own quality assessment before committing to anything. With proprietary solutions, you often have to trust the vendor much earlier. Sometimes, that feels like buying a pig in a poke.

Recommendations

At a high level, we can group Gateway API implementations by the data plane they use:

Envoy Gateway

Envoy Gateway is an open source Gateway API controller based on Envoy Proxy and developed by the Envoy Proxy team.

The goal is to map Envoy Proxy capabilities into Gateway API as directly as possible. That makes Envoy Gateway easier to understand because there are fewer product-specific abstractions compared to something like Istio.

It does not support the Ingress API, but it can be extended with Envoy AI Gateway, which adds LLM gateway, MCP gateway, and Inference Pools capabilities.

Envoy Gateway is lightweight and straightforward to deploy and operate. It is laser-focused on API Gateway, or North-South traffic, concerns. It supports:

  • security patterns like external authentication, JWT validation, JWT claim-based authorization, OIDC, IP allowlisting, static API key authentication, and credential injection
  • flexible global rate limit policies with global and route-level configuration, shadow mode, request cost configuration, and other controls
  • connection, bandwidth, and file size limits
  • availability zone-aware routing
  • basic multi-cluster support via ServiceImport
  • response compression with Brotli, gzip, and zstd
  • direct responses and response overrides

Envoy Gateway is also very extensible:

  • You can create a gRPC service to modify requests, responses, or the xDS configuration produced by the control plane.
  • You can write plugins in Lua or anything that can compile to WASM, such as Go or Rust.

Envoy Gateway includes a custom Backend resource that lets you target additional backend types, such as:

  • external resources by FQDN or IP address
  • UNIX domain sockets for low-latency local or sidecar service processes

Envoy Gateway is currently listed as a partially conformant due a few skipped tests, even though functionally it ticks almost all the boxes.

The ecosystem also includes integrations with other CNCF projects like KEDA and KNative.

If you need a feature-rich API Gateway and nothing else, Envoy Gateway is a great option.

Istio

Istio is a service mesh based on Envoy Proxy and maintained by the CNCF as a Graduated project.

It is listed as conformant with the Gateway API test suite.

Istio is a whole package that includes:

  • an Ingress Controller
  • a Gateway API controller
  • a service mesh, which is what it is best known for

It can also be integrated with agentgateway to provide LLM, MCP, and A2A gateway capabilities.

Istio also has advanced multi-cluster capabilities, especially with projects like Admiral.

Istio has a wide deployment profile, a large community, and many learning resources, including O’Reilly books. It is also known to be useful in government and FedRAMP-style environments because its service mesh can provide pod-to-pod encryption with mTLS.

So it is a very battle-tested option.

However, Istio is also famously heavy in a few ways:

  • It can be resource-heavy, especially if you run it in sidecar mode.
  • It has a real learning curve because it comes with a lot of functionality and its own concepts.

Istio has ambient mode, which is meant to provide a lighter service mesh setup. However, ambient mode is not always as feature-rich as sidecar mode for advanced L7 use cases. If you need stronger Gateway API policies or richer Envoy-based L7 controls, you may also consider using Envoy Gateway as an ingress gateway or waypoint proxy.

Istio is the best pick when you need service mesh capabilities first and Gateway API support second. That way, you can fully use the power of Istio and its ecosystem.

A Gateway API-only installation is possible, but Istio still revolves around service mesh use cases. If all you need is an API gateway, you may feel a bit underserved.

kgateway

kgateway is an open source, CNCF (Sandbox) gateway based on the Gloo project.

It gives you control over the data plane. Today, it supports two main options:

You can also bring your own data plane instance managed somewhere else, which is an interesting feature.

kgateway has perfect Gateway API conformance. Nearly the only implementation that ticks all checkboxes on the conformance current suite.

With the Envoy data plane, kgateway exposes useful Envoy Proxy capabilities, such as:

  • security patterns like JWT validation, OAuth2/OIDC integration, external authentication, and IP access control
  • Envoy’s global rate limiting
  • request and response processing through Envoy’s ext_proc extension point

However, it does not seem to expose Envoy’s Lua or WASM plugins, or the ability to modify raw xDS configuration directly.

kgateway also has a powerful transformation engine based on the MiniJinja template language. This lets you declaratively define header, response body, and status transformations directly in your TrafficPolicy resources.

kgateway can be integrated with istio as an edge proxy for ingress traffic, but it does not act as a service mesh on its own.

kgateway supports hierarchical Routess. The idea is that one Route can delegate part of its traffic handling to another Route. This can be useful for complex route organization and ownership boundaries.

It also has a custom AwsLambda CRD that lets you invoke AWS Lambda functions directly from the gateway.

Overall, kgateway looks like an interesting option.

That said, I have not found much independent feedback about it yet, despite vendor claims around wide deployment. That makes me think it is still a growing project from the public community perspective.

If you use kgateway in production and are willing to share your experience, let me know.

Traefik

Traefik is a popular open source reverse proxy written in Go. It supports both the Ingress API and Gateway API.

I think it owes its popularity to great documentation, a well-organized codebase, straightforward configuration, and easy deployment. It is simple and ergonomic. That is why many people love it, especially in the homelab community, where you usually want to have a good developer experience and don’t have any enterprise requirements that stands in your way.

Traefik supports core Gateway API functionality and some extras). For example, it does not yet support ListenerSet or traffic mirroring through Gateway API, although it supports traffic mirroring in general through custom TraefikService backend resources.

Traefik’s extension model is based on middleware. It actively uses the ExtensionRef filter to point Routes to the Middleware CRD. There are several built-in middlewares, including:

  • ForwardAuth, which delegates authentication, and potentially authorization, to an external service. It is similar to Envoy’s ext_authz extension.
  • IP allowlisting and CORS
  • connection limits, retries, and circuit breakers
  • compression and custom error pages

The Plugin middleware lets you attach custom YAEGI and WASM plugins to Routes. These plugins mostly focus on request and response modifications.

JWT validation, OIDC, and OAuth2 Client Credentials support are only available through paid plugins.

Traefik supports basic distributed rate limiting through Middleware CRDs. You can bucket rate limits by IP, host, or headers. However, the configuration does not seem very flexible. Since rate limiting is attached to Routes through the ExtensionRef filter, rather than through policy attachment, there is no easy way to layer policies, apply one policy globally, and override it at the route level.

Traefik also does not have a clear control/data plane split. Its architecture is closer to NGINX Ingress: the same pod watches Kubernetes resources and serves traffic. It also does not dynamically provision separate proxies per Gateway resource. Instead, one deployment manages all Gateway resources in the namespaces it is configured to watch. That can make Traefik a single point of failure and reduce isolation between Gateways, which may become a resiliency problem at scale.

Finally, if you want to pick Traefik, make sure you load test it against your expected traffic profile. This advice applies to any gateway implementation, but I have heard performance complaints about Traefik specifically, so I would be extra careful here.

NGINX Gateway Fabric

NGINX Gateway Fabric, or NGF, is F5’s official Gateway API implementation based on NGINX.

NGF has solid Gateway API conformance. Recent versions added support for Gateway API 1.5, CORS and request/response header modification through standard HTTPRoute filters.

Some functionality is still tied to NGINX Plus, including:

  • JWT and OIDC authentication
  • cookie-based session persistence
  • upstream backend updates without NGINX reloads

That matters because these are not exotic enterprise-only features. They are common API Gateway requirements.

NGF also comes with custom extension resources like SnippetsPolicy and SnippetsFilter. They let you inject custom NGINX configuration at different levels of the generated config. This can make a Gateway API migration easier if you already have a ton of custom NGINX configuration in your NGINX Ingress Controller setup and do not want to rewrite them all.

NGF supports rate limiting through the custom RateLimitPolicy resource. One important limitation is that this is local rate limiting. The rate-limit state lives in the NGINX data plane, not in a shared global service. That makes it harder to enforce strict user-level limits when you run multiple NGF replicas, because the effective limit may change with the number of data plane instances.

SnippetsPolicy, SnippetsFilter, and RateLimitPolicy are examples of NGF’s custom extension model. On top of that, you can extend NGINX itself through NGINX snippet escape hatch:

  • lightweight JavaScript or Lua scripting, if the corresponding modules are available in your NGINX image.
  • The auth_request directive, which can delegate external authentication to another service, similar to Envoy’s ext_authz or Traefik’s ForwardAuth middleware.
  • Custom NGINX modules, which is the most involved option. It requires a custom NGINX build and knowledge of C.

NGF does not have a simple ext_proc-style way to modify the request/response flow through an external service.

Finally, NGF 2.0 reworked the architecture to provide a proper control/data plane split and support multiple Gateway resources. Before that, the architecture was a big concern.

Cilium Service Mesh

Many Kubernetes clusters already use Cilium as their CNI.

Cilium also comes with an eBPF-based, sidecar-less service mesh, and that service mesh includes a Gateway API implementation based on Envoy Proxy. So it might be tempting to just use what Cilium already provides as your API Gateway. That gives you fewer moving parts and a slimmer tech stack to maintain.

In terms of functionality, though, Cilium Gateway API seems mostly focused on Gateway API conformance. A lot of useful Envoy Proxy functionality that is not part of Gateway API is not surfaced as first-class Cilium API Gateway configuration yet. For example, I do not see first-class support for:

  • Envoy extensions or plugins
  • IP allowlisting
  • JWT validation, claim-based authorization, OIDC

So unless you are confident that Cilium’s current Gateway API functionality is all you need, I would not pick Cilium Service Mesh as a general-purpose API Gateway over more feature-rich options like Envoy Gateway, kgateway, or Istio.

Kong

Kong Gateway is a fairly popular API Gateway based on NGINX. Kong Operator supports both the Ingress API and Gateway API.

The main concern I see with Kong is around its OSS story.

Kong appears to have stopped publishing prebuilt Docker images for newer open source Gateway versions. For example, users noticed that OSS images stopped around the 3.10 release line, which means you may need to build, patch, harden, and maintain the image yourself.

That sparked public speculation that this move is related to reducing enterprise customer churn. I would not treat that as a confirmed fact, but it does make the direction of Kong’s OSS offering feel less predictable.

Because of that, I would not pick Kong unless you are ready to buy the enterprise license or are comfortable owning more of the OSS packaging and maintenance story yourself.

Summary

We have covered a lot of ground here.

We traced the evolution of Kubernetes ingress patterns from the early days of Kubernetes to the Gateway API era. Then we stopped at Gateway API and went deeper into the protocol itself.

I intentionally left a few important Gateway API extensions out of scope, namely GAMMA and Inference Extension. GAMMA extends Gateway API ideas into the service mesh space. Inference Extension focuses on defining and optimizing inference workloads in Kubernetes. Both are worth knowing about, but they deserve separate deep dives.

Finally, we looked at available Gateway API implementations and how to think about picking one.

I hope this article helps you make your Gateway API migration smoother and your planning more informed.

References