Written by Marcin Krzyszkowski
Software Engineer
Published May 16, 2023

Should you run your own API Gateway? – Spring Cloud Gateway

 

Before answering the question above, let’s see why having an API gateway is essential in a modern, microservice-based infrastructure. A quick Google search for “what’s an API gateway” will yield answers to the tune of:

An API gateway is an API management tool put in place between clients and the backend services they need to access. The API gateway acts as a reverse proxy accepting all incoming application programming interface (API) calls, aggregating the various services required to fulfil them, and returning the appropriate result.

In other words, an API gateway serves to decouple the client interface from the backend implementation. In its simplest form, a gateway would accept an incoming request, route it according to configured rules and just return the received response. Most enterprise APIs, however, require gateways to handle common tasks accompanying accessing their APIs. Precisely what the API gateway does will vary from one implementation to another, but this can include authentication, rate limiting, billing, monitoring, analytics, etc. And as API complexity increases and usage grows, so do the responsibilities and the value of an API gateway.

What are some pros and cons of using an API gateway?

Based on what we’ve already learned, the benefits and drawbacks will vary based on the needs of a given project. However, in general, we can at least identify the following:

Benefits

  • obfuscates how the application is partitioned into microservices from the clients
  • allows for performing common tasks (e.g. authentication, rate limiting, …) in an automated, unified manner
  • reduces the number of requests/round-trips if there is a possibility of data aggregation
  • translates from a public, client-friendly API protocol to whatever protocols are used internally

Drawbacks

  • increases complexity as it is yet another moving part that must be developed, deployed and managed
  • increases response time due to the additional network hop (for most deployments, the cost of an extra hop is negligible)

Our team’s decision

At this point, one may look around and see ready-to-use offerings, like, for example, those present in cloud providers’ portfolios – Amazon API Gateway or Google Cloud API Gateway from AWS and GCP, respectively. Or, for an even more general solution, look to Kubernetes Gateway API, available to anyone running their services in a Kubernetes cluster.

Services such as those listed above have the benefit of being relatively easy to get up and running quickly with minimal amount of additional configuration while also providing powerful mechanisms for authentication, monitoring, rate limiting and alike. Unfortunately, at the same time, in certain use cases, they tend not to give enough flexibility as we would be able to have when running our own solution.

Knowing the needs of a given project should help decide on the best approach. Our team’s product is being developed as a part of Schibsted’s Nordic Marketplaces offering and needs to be accessible by customers visiting the FINN, Blocket or Oikotie marketplaces. There is a catch – while each marketplace has its own infrastructure, internal services and authorization schemes, our product must be available as part of one common platform and run “outside” of each marketplace’s environment. At the same time, this decoupling must be transparent to customers using our product. Unfortunately, even though our services are being hosted in Google Cloud, their gateway offering is not one of the services made available to product teams. Moreover, while we run our applications on Kubernetes, we do it through an internal IaaS (infrastructure as a service) solution on top of Kubernetes, which means we also can’t use its gateway without interfering with services from other teams. Thus, the specifics of our use case necessitated implementing and deploying our own solution…

Spring Cloud Gateway

Considering that our backend team consists of Kotlin/Java developers and our services run on Spring Boot while making heavy use of other Spring libraries, an offering of theirs – Spring Cloud Gateway (reference), seems perfect for the task. Being a part of Spring’s ecosystem, Spring Cloud Gateway takes advantage of the developer’s familiarity with Spring’s development model. Most of what you can do in a typical Spring application, you can also achieve in an application running on Spring Cloud Gateway. Seeing as it has been built on top of Spring Boot and Spring WebFlux and is using a reactive stack, the main limiting factor to what can be achieved is the compatibility of widely available synchronous libraries and patterns with such an approach.

How does it work?

First, let’s introduce a few terms key to understanding how the gateway does what it does. The main building block of the gateway is called a route. It consists of a collection of predicates, a collection of filters, and a destination URI. Next, we have a predicate. This is just a Java Predicate which takes Spring’s ServerWebExchange as a parameter. That means we can match any parts of the HTTP request (path, headers, parameters, etc.). Lastly, there is a gateway filter. Filters modify the request and response as it passes through the gateway’s filter chain. We identify two types of filters – “pre” filters, which are being run before the client’s request is proxied to the destination service, and “post” filters, which are being executed on the way back after the proxied service has responded.

To put those terms into some context, let’s look at the diagram below:

When a client makes a request to an application running Spring Cloud Gateway, it first goes through the Gateway Handler Mapping to determine whether it matches any known routes and their predicates. The request is then sent to Gateway Web Handler, which executes a filter chain containing all “pre” filters appropriate for the matched route, and upon reaching the end of the filter chain, sends off the request to be handled by some service. Once a response from that service has been received, the Gateway Web Handler executes the rest of the filter chain with its “post” filters. Finally, the response is returned to the client which has made the request.

Setup

Let’s see how it all looks in practice. To create our project, we’ll be using Kotlin with Gradle. As mentioned previously, an application running Spring Cloud Gateway is just like any other Spring Boot application. That means we can use just the starter provided by Spring Cloud Gateway project, and all required dependencies will already be there. And so, our build.gradle.kts file will look as simple as:

plugins {
    kotlin("jvm") version "1.7.22"
    kotlin("plugin.spring") version "1.7.22"
    id("org.springframework.boot") version "3.0.6"
    id("io.spring.dependency-management") version "1.1.0"
}

val springCloudVersion = "2022.0.2"

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.springframework.cloud:spring-cloud-starter-gateway")
}

dependencyManagement {
    imports {
        mavenBom("org.springframework.cloud:spring-cloud-dependencies:$springCloudVersion")
    }
}

Here, we’re specifying the version of Spring Cloud’s release train, in our case 2022.0.2, and Spring’s dependency management plugin pulls in the version of Spring Cloud Gateway appropriate for that release (4.0.4).

Next, we create our application’s main class and run it to see if everything is working:

@SpringBootApplication
class GatewayApplication

fun main(args: Array<String>) {
    runApplication<GatewayApplication>(*args)
}

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.0.6)

...  INFO --- [main] c.s.cmt.gateway.GatewayApplicationKt     : Starting GatewayApplicationKt using Java 17.0.3 with PID 18903 (...)
... DEBUG --- [main] c.s.cmt.gateway.GatewayApplicationKt     : Running with Spring Boot v3.0.6, Spring v6.0.8
...  INFO --- [main] c.s.cmt.gateway.GatewayApplicationKt     : The following 1 profile is active: "local"
...  INFO --- [main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=743a3f26-97b1-3444-998f-282720f0bbbe
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [After]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Before]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Between]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Cookie]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Header]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Host]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Method]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Path]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Query]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [ReadBody]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [RemoteAddr]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [XForwardedRemoteAddr]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Weight]
...  INFO --- [main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [CloudFoundryRouteService]
...  INFO --- [main] o.s.b.web.embedded.netty.NettyWebServer  : Netty started on port 8080
...  INFO --- [main] c.s.cmt.gateway.GatewayApplicationKt     : Started GatewayApplicationKt in 0.875 seconds (process running for 1.111)

Great, the application is up and running, and we see that it uses Spring Cloud Gateway. It has also loaded some default predicates. However, we have not defined any routes, so while we can receive requests, the gateway won’t proxy them anywhere.

Let’s fix that. We can do it in two ways – either through configuration properties in Spring’s application.yml file or code, using a builder provided by the framework. So, in the YAML file, our configuration could look like this:

spring:
  cloud:
    gateway:
      routes:
        - id: test-route
          uri: https://httpbin.org
          predicates:
            - Path=/test/**
          filters:
            - SetPath=/headers
            - AddRequestHeader=X-Gateway-Routed, test-route
        - id: catch-all-route
          uri: https://httpbin.org
          predicates:
            - Path=/**
          filters:
            - SetPath=/headers
            - AddRequestHeader=X-Gateway-Routed, catch-all-route

And in Kotlin:

@Service
class GatewayService {

    @Bean
    fun routes(builder: RouteLocatorBuilder) = builder.routes {
        route("test-route") {
            path("/test/**")
            filters {
                setPath("/headers")
                addRequestHeader("X-Gateway-Routed", "test-route")
            }
            uri("https://httpbin.org")
        }
        route("catch-all-route") {
            path("/**")
            filters {
                setPath("/headers")
                addRequestHeader("X-Gateway-Routed", "catch-all-route")
            }
            uri("https://httpbin.org")
        }
    }
}

So, what exactly happens here? We’ve defined two routes - test-route and catch-all-route. The first uses a Path predicate and is set up to match all requests with path prefix /test. The second route uses a wildcard /** pattern, which will be matched by any request the other route hasn’t received. Unless a route explicitly sets an order property, all routes are evaluated in top-to-bottom order. We’ve also added two filters to both paths, SetPath and AddRequestHeader, which, as their names suggest, will first change the path and then add a header to all requests passing through a given route. We’re using httpbin.org‘s headers endpoint to test our gateway and see what has been forwarded downstream. Running the application and calling the defined routes will yield the following responses:

$ curl http://localhost:8080/test
{
  "headers": {
    ...
    "X-Gateway-Routed": "test-route"
  }
}

$ curl http://localhost:8080/not-a-test
{
  "headers": {
    ...
    "X-Gateway-Routed": "catch-all-route"
  }
}

Perfect! The routing works, and depending on the path we’ve called, a different header was passed to the downstream service. And that’s it – we now have an application running on Spring Cloud Gateway that we can extend with different routes to match our needs.

As a side note, it is worth mentioning that the DSL used in the code is Kotlin-specific. If our project used Java, we’d still be able to achieve the same configuration as here; we would just use a standard builder – not as easy on the eyes.

Expanding our gateway

REST controllers

Let’s see what more we can do with our gateway. It’s been said that a Spring Cloud Gateway application, in many ways, is just like any other Spring Boot application. Does that mean, for example, that we can create regular REST controllers and have them work alongside the routes we’ve defined? Yes, absolutely! One thing to remember, though, is that endpoints defined in classes annotated with @Controller and @RestController take precedence over any Spring Cloud Gateway routes. If we add the following code to our application without changing anything else we’ve done so far, we’ll receive a response from the controller, not from the test-route:

@RestController
class GatewayController {

    @GetMapping("/test")
    fun test(): String = "ok from controller"
}

$ curl http://localhost:8080/test
ok from controller

Custom predicates

Spring Cloud Gateway has a set of built-in predicates, allowing us to create quite granular route-matching logic. This includes matching on request HTTP method, path, headers, cookies, query parameters and more (complete list). There are times, however, when that is simply not enough. That’s where Spring Cloud Gateway shines because it allows us to implement our own predicates that execute whatever logic we require. Let’s see how we could implement a predicate that uses an external authorization service to restrict access to our APIs.

First, we extend AbstractRoutePredicateFactory and implement its apply method:

@Component
class AuthorizedPredicateFactory(
    private val authService: AuthService
) : AbstractRoutePredicateFactory<Config>(Config::class.java) {

    override fun apply(config: Config): Predicate<ServerWebExchange> {
        return GatewayPredicate { exchange ->
            exchange.request.headers[config.header]
                ?.firstOrNull()
                ?.let {
                    authService.authorize(it)
                } ?: false
        }
    }

    data class Config(val header: String)
}

Here, we’re accepting a header name parameter in a Config parameter and extracting that header’s value from ServerWebExchange's request. We then call some authorization service and return its result (true, false) from the predicate.

For us to be able to use the AuthorizedPredicateFactory we need to modify the GatewayService by adding an autowired context parameter and the following method:

@Service
class GatewayService(
    private val context: ConfigurableApplicationContext
) {
    ...

    fun PredicateSpec.authorized(authHeader: String): BooleanSpec {
        return asyncPredicate(
            context.getBean(AuthorizedPredicateFactory::class.java)
                .applyAsync(AuthorizedPredicateFactory.Config(authHeader))
        )
    }
}

By creating the authorized function as an extension method on Spring Cloud Gateway’s PredicateSpec, we can use it directly in the DSL and chain it nicely with other predicates. We can now modify the code of one of our routes and have it secured by our authentication service:

route("test-route") {
    path("/test/**") and authorized("X-Gateway-Auth")
    filters {
        setPath("/headers")
        addRequestHeader("X-Gateway-Routed", "test-route")
    }
    uri("https://httpbin.org")
}

From this point on, only requests containing a X-Gateway-Auth header with a valid value would be matched by test-route route.

Custom filters

Now that we’ve secured our API, we may want to apply some filters and modify requests or responses passing through our gateway. As with predicates, Spring Cloud Gateway provides multiple useful filters to suit one’s needs. A complete list can be found here. Some notable implementations include filters for modifying request and response headers, body, and parameters, manipulating the request’s path, managing timeouts, caching and so on.

We’ve learned previously that filters can be divided into two groups, “pre” and “post”. Although both types can be created by extending AbstractGatewayFilterFactory, the implementation differs slightly between the two. Let’s create one of each and see if we can spot the differences:

@Component
class AddSessionGatewayFilterFactory(
    private val sessionService: SessionService
) : AbstractGatewayFilterFactory<Config>(Config::class.java) {

    override fun apply(config: Config): GatewayFilter {
        return GatewayFilter { exchange, chain ->
            val sessionId = exchange.request.headers[config.header]
                ?.firstOrNull()
                ?.let {
                    sessionService.getSession(it)
                }
                ?: throw IllegalStateException()

            sessionId.map { id ->
                exchange.request.mutate()
                    .header("X-Gateway-Session", id)
                    .build()
            }
            .flatMap { request ->
                chain.filter(exchange.mutate().request(request).build())
            }
        }
    }

    class Config(val header: String)
}

@Component
class SetLanguageGatewayFilterFactory(
    private val languageService: LanguageService
) : AbstractGatewayFilterFactory<Config>(Config::class.java) {

    override fun apply(config: Config): GatewayFilter {
        return GatewayFilter { exchange, chain ->
            chain.filter(exchange).then(Mono.fromRunnable {
                val languageCookie = languageService.getLanguageCookie()
                exchange.response.addCookie(languageCookie)
            })
        }
    }

    class Config
}

In the case of “pre” filters, we modify the request being proxied to the destination service. This is why our logic comes before chain.filter(...). In the AddSessionGatewayFilterFactory above, first, we retrieve the value of our authorization header and then use it to get a session identifier from a SessionService existing somewhere in our codebase. Because our session service is blocking, the getSession method returns Mono<String> from Project Reactor used by Spring WebFlux. Whenever we call such code in our filters, we need to make sure we are chaining the actions using a reactive approach. So, in our case, we take the sessionId we’ve retrieved and mutate the request by adding a new header to it – X-Gateway-Session. Finally, we can take the resulting request and mutate the whole ServerWebExchange and pass it along via filter method of the gateway’s filter chain.

With “post” filters, we modify the response that is on its way back to the calling client. We need to first call the chain.filter(...) method, and only then we can execute our own code. We still have access to the request passed along, but modifying it makes no sense at this stage. We could use it, however, to retrieve some parameters from it. In the SetLanguageGatewayFilterFactory, we’ve created a filter that adds a language cookie to the response meant for the client, possibly a web browser.

To use both of our newly created filters in the application, we need to add the necessary methods to the GatewayService:

fun GatewayFilterSpec.addSession(authHeader: String): GatewayFilterSpec {
    return filter(
        context.getBean(AddSessionGatewayFilterFactory::class.java)
            .apply(AddSessionGatewayFilterFactory.Config(authHeader))
    )
}

fun GatewayFilterSpec.setLanguage(): GatewayFilterSpec {
    return filter(
        context.getBean(SetLanguageGatewayFilterFactory::class.java)
            .apply {}
    )
}

Again, we implement them as extension methods, this time on GatewayFilterSpec, so we can use them without hesitation. Notice that in the case of the SetLanguageGatewayFilterFactory we didn’t need any config parameters, so we leave the apply method empty.

And the final modification of our test-route:

route("test-route") {
    path("/test/**") and authorized("X-Gateway-Auth")
    filters {
        setPath("/headers")
        addRequestHeader("X-Gateway-Routed", "test-route")
        addSession("X-Gateway-Auth")
        setLanguage()
    }
    uri("https://httpbin.org")
}

Testing

When it comes to testing our Spring Cloud Gateway-specific code, what we’ve found works well for our team, is to unit test the predicates and filter logic and then have an integration test for the GatewayService, via @SpringBootTest web environment, where we can use Spring’s WebTestClient in conjunction with any mock web server to check whether all predicates and filters behave correctly together, in the routes we’ve defined.

Conclusion

The answer to the question – “Should you run your own API gateway?” is not definitive. As mentioned multiple times throughout this text, the need to do so will depend entirely on the circumstances and complexity of the project. There are strong incentives to run your custom solution based on, for example, Spring Cloud Gateway. We’ve seen how it may be beneficial when operating within an already well-established ecosystem, needing to perform multiple custom actions for every request/response or having to deal with exotic authentication logic. Deploying a self-developed solution provides complete control over what happens to incoming requests and outgoing responses.

In the case of our team’s product, it was the best tool for the job. We are thrilled with how our Spring Cloud Gateway based solution performs and its (almost) limitless possibilities. With that in mind, we encourage all of you to try it out and see if it has a place in your projects.

 

Written by Marcin Krzyszkowski
Software Engineer
Published May 16, 2023