May 23, 2020

Real world API ecosystem architecture

In the post about APIs and SOA, I mentioned briefly the architecture we designed and built to the support MAD-API: the API ecosystem.

To begin understanding MAD-API and API Management, the first starting point is the 3 API layer design: API-Led connectivity. MuleSoft's API driven approach, and Gartners Pace Layering, (see here and here) defines three fundamental types of APIs that should be created:

API layering
  • Experience APIs for each consuming client
  • Process APIs for orchestration
  • System APIs for system of record integration

This resulted on the complete MAD-API Platform looking like:

MAD-API

The API Management Platform, in our case Google Apigee, served as the Experience layer. It exposed APIs on the dev portal, controlled auth, and provided stats, security, rate limiting, etc, out of the box.
The bottom two API layers are written in Spring, deployed as containers to k8s, via a pipeline. This allowed us to re-use layers across different OpCos E.g. the mainframe system layer written in one OpCo, could be used re-used in another OpCo by just pulling and deploying that layer, but could use a different process layer on-top of it.
By separating each distinct function to its own module, allowed multiple developers to work on different layers at the same time to build a completed API.

API Design

We tried to follow good API design, specifically using OAS/Swagger to describe the API.
Different layers communicated primarily via synchronous APIs, which later on became a bottleneck, because we realised we needed to use different architecture styles for different use cases. Later on we tried introducing an event-based messaging system.
We later on also realised that we couldn't always rely on querying the system of records synchronously, so we needed to introduce a DB to store/cache certain data, in order to meet latency requirements.

DevOps

We tried to make sure that deployment to each component could be done via the pipeline. So a developer could push code, which will deploy to Apigee, k8s, etc. Developers could start off with using the template generated by the initialiser, which will upfront create the git repo, pipeline, proxies, k8s deployments - before each writing a single line of code.

Dogma

Like with all things in life, especially in tech, its to become dogmatic and 'strictly' follow a certain pattern - because we claim it makes our life easier to manage by standardising. Standardisation can easily lead to the Lowest Common Denominator - the simplest and lowest level of commonality, which means the lowest support for features.

The close this post off, I'll talk about something a bit different. In order to share and advocate for MAD-API, we held a global workshop, where different OpCos flew in for the week, to discuss how the platform can enable them. For make things a bit fun, we printed T-shirts and caps, and that really made the MAD-API brand very real.

Tshirts and caps for the workshops