How to integrate your AWS event-based system with external/legacy systems using RESTful APIs
To successfully implement an API-led organization necessary for Digital Transformation requires more than just technology – it requires a transformation of software development capabilities
Case studies of telcos such as Telstra, KPN and adjacent industries show that the leaders are leveraging inhouse Agile DevOps capabilities in order to rapidly deliver differentiated digital customer experiences.
TLDR: Without in-sourcing development of key capabilities, your digital ambitions will remain a pipe dream.
The Modern Application Delivery Playbook
Taken from Forrester's report, most IT organizations don't have the right operational models to develop and deliver great digital experiences. This report makes a case that digital and software development leaders should take a customer-focused approach to software development in support of digital transformation efforts.
Lets start with some background and context before we get into the meat of the topic. Many large businesses, in telco and other industries, have largely, over the last decade, outsourced parts or all of their development capabilities. Specifically for Telcos, by using vendors like Ericsson, Huawei, ZTE, etc, in areas like SDP, Billing, Charging/IN, CRM, and other key platforms, have outsourced the development of these platforms to the off-shore central development hubs of the vendors, usually based in China or India, far away from business. That makes business mostly dependant on the feature roadmap of the platform, dictated by the vendors. When any customisation of the platform is required, it typically follows a very long, drawn-out waterfall SDLC, where they use requirements documents to communicate (we will contrast this to agile, which identifies that humans need to actually speak to each other to understand each other). After some time, and many documents later that capture the requirement in overly complex ways, with the sole purpose to make it easy for some remote developer in China to understand, without having to speak to business at all. After some time (1 - 3 months), the code comes back from, and business now, waiting anxiously and impatiently, open it up like a gift, hoping that its actually what they asked (read wished) for. After some waiting for it to be implemented into production, business finds that it (usually) isn't completely what was asked for, so back again goes the requirements docs, and again we wait for a few months. Rinse and repeat, and business knows they cannot respond quickly enough to users and markets needs. They shout and demand for better support from IT, who usually just blame the vendor, the platform, or both. They might go out on RFP to replace the vendor, the platform, or both. Most will not even consider that their problems lie with their over-reliance on out-sourced vendors, dogmatic compliance to archaic processes like Architecture Review Boards (ARB) and CAB, and outsourcing: which is fuelled by their never-ending quest of costs savings (a myth) and their need for "one throat to choke" insanity.
In the end, we can summarise it as: Businesses and IT are struggling to meet the needs and demands of their customers, and simply replacing the old monolithic platform is not enough. The process and cultural issues run much deeper, and need to be addressed, if we ever going to meet our digital transformation targets.
The lessons learned from using a waterfall approach to development
- Business ideas to requirements are documented in some requirements document (URS, RDS, etc) that is sequentially reviewed which is a very time-consuming process
- Once the technical design (FRS, TRS) is completed, developers have already completed the design and construction stages of development, preventing the addition of a new feature until the next project planning stage which occurs in 3 months time
- Development and Operations teams are adversaries – development want to push new features, but Operations are too slow to catch up
- Intensive and manual Testing activates happen only at the end of the project, too late to correct any problems found
- Any critical activity requires too much manual effort and too many handoffs.
- This leads to extremely long lead times to get anything done, but the quality of our the work, especially production deployments is also problematic and chaotic, resulting in negative impacts to our customers and our business.
Moving towards modern software practices
In todays rapidly evolving Application Economy, it is widely recognized that, driven by evolution of digital channels, many organizations are reinventing themselves as providers of software an digital services. As traditional business services are replaced by largely web and mobile applications, organizations are being forced to completely reexamine their software development and IT management practices.
Technology is no longer viewed as a supporting dependency, but rather as the primary element of conducting business.
Within that context, most of todays organizations are moving aggressively to adopt more agile, efficient software delivery practices to meet customers evolving expectations. Amongst the fastest growing and most widely adopted strategies, aimed at bringing Toyota-like efficiency to the world of applications lifecycle, is the methodology knows as DevOps.
A Bi-modal IT architecture for the digital enterprise
Two-speed architecture could help companies balance resources more appropriately. Investing in new technologies, software-development approaches, and top development talent can help companies differentiate themselves from competitors. But it may be expensive to do these things across all business units. The very exercise of building a two-speed architecture can help companies set priorities.
A two-speed IT architecture will help companies develop their digital capabilities at high speed while decoupling legacy systems for which release cycles of new functionality stay at a slower pace. This implies a fast-speed, customer-centric front-end running alongside a slow-speed, transaction-focused legacy back-end. For software release cycles and deployment mechanisms, the digital customer-facing part should be modular, to enable quick deployment of new software by avoiding time-consuming integration work. In contrast, the transactional core systems of record must be designed for stability and high-quality data management, which leads to longer release cycles.
Why Software Is Eating The World, but requires Focus
Gartner: By 2022, more than 50% of midsize, large and global organizations will establish an integration strategy empowerment team to support a self-service integration approach.
Data like this shows that an investment in a software stock yields more than Telecoms stocks.
Technology – software in particular – has had a destabilizing effect on traditional business models. The proliferation of personal computing power has leveled the playing field in almost every industry. As products and the means to create them have become digitized (often referred to as software eating the world), production capability has grown more accessible and portable.
According to the U.S. Bureau of Labor Statistics, software developer jobs are expected to grow 17% from 2014 till 2024. They categorize this growth as “much faster” than the average rate among other professions.
Here is a quick summary of the famous article written in 2011 by Marc Andreessen
- Today, the worlds largest bookseller, Amazon, is a software company
- Today, the largest video service by number of subscribers, is software company: Netflix
- Todays dominant music companies are software companies: Apples iTunes, Spotify, Pandora.
- Todays fastest growing entertainment companies are video game makers, growing to $60 billion from $30 billion five years ago
- Todays fastest growing telecom company is Skype – a software company bought by Microsoft for $8 billion
It turns out that it’s not just software that’s’ eating the world. Firms are learning the hard way that software requires a different way of running the organization to be successful. Firms have to be nimble, adaptable, able to adjust on the fly to meet the shifting whims of a marketplace driven by the customer. This kind of management was—and is—beyond the capabilities of the lumbering industrial giants of the 20th Century. It is firms that are truly agile that are eating the world
Insource your core - augment your context
Why do companies like Amazon, Netflix, Stripe, and Uber value software more than any other corporate asset? Because they understand that software is the vehicle that drives digital business. It allows them to deliver unique experiences for their customers, it automates processes, and it enables them to assess and access new platforms, frameworks and systems that will speed the prototyping and testing of new ideas.
It’s hard to be a software-driven company without your own talent. But as-a-service platforms and open source allow teams to write a lot less code today compared to previous generations of applications. To move fast, digital leaders should build the services, user experience, and algorithms that create unique customer value with in- house development teams, augmented with external talent as needed.
Systems that do not create unique customer value (context) are a better candidate for outsourcing, purchasing, extending, or configuring on top of another firm’s software assets and talent
Focus on your differentiators
The core application and ecosystem around the Uber experience is their primary asset and differentiator:
- They run on infrastructure provided by Amazon, so they can be up and available all around the world.
- Their mapping technology is provided by Google Maps in the form of an API, so drivers take the fastest routes and you always know where you are.
- Their messaging stack is provided by Twilio, ensuring you get that text message right when your driver has arrived.
- And their email service to send out things such as receipts to passengers is built on SendGrid APIs.
Uber leaves all of this to specialists who focus on these elements as their respective core businesses. What Uber instead focuses on is curating a world-class experience for their customers and solving the problem of transportation for the masses, one ride at a time.
Market research - Success Stories
Here are some quick one liners, showing how the market values software excellence:
Lets let at some details, showing more of how big companies have embarked on this journey
This powerful article explains how KPN reintegrated previously outsourced expertise and shifted to agile ways of working.
Lets look at some key highlights:
KPN Digital: A number of small autonomous Agile teams have built up the digital products and online customer systems from the ground up, with intensive use of Cloud, CI/CD as standards, and DevOps mindset
KPN realized that they were competing not just with other telcos but also digital-native companies. Their attempts to engage our customers drove up our customers’ expectations about our own digital capabilities. They were spending a lot of money trying to keep up, but we continued to lack speed, flexibility, and a strong customer focus. They were overly dependent on our sourcing partners; a lot of our resources were located on a different continent, far away from the business. In the meantime, they were often delivering services over budget and not always meeting the needs of the business. They decided the only way to change this was to look at agile methodologies, and part of that was insourcing knowledge and bringing the developers closer to the business again.
KPN AppFactory: A KPN Service that bundles practices and tool chains in the field of CI/CD, Infrastructure as Code, and Containers, delivered to their customers Agile teams.
They were able to introduce one KPN identity for all mass-market KPN customers, so they did not need to manage multiple log-ins. KPN developed a “digital engine” that acts as a single-interface layer between the old telco and the new digital channels and applications. This engine makes about 300 traditional services available by generic APIs to every developer, regardless of which area of IT they specialize in
Digital IT has one HR head, four development coaches, a finance manager, and a tech lead, and that’s it. He oversee 30 scrum teams without further formal hierarchy. They have our small “vision” team that sets strategy and guides all development efforts. The key to our success, beyond structure, has been communication and culture change
KPN moved all of our applications to digital open-source technology, which helped us cut our run rate in half while delivering the same output. Their overhead is lower, and they have automated virtually everything, so their developers can spend about 90 percent of their time on software rather than on other non-value-adding tasks.
Domino’s Pizza CEO Patrick Doyle has been talking about being a tech company that sells pizzas for years
Lets share some quotes from this article about Domino's technology dominance:
Domino’s, however, remained relentless, forging ahead on innovations and leaving independents and smaller chains in its dust. In its dust they remain, simply unable to keep up with the technology infrastructure Domino’s has amassed.
The approach has led to 28 consecutive quarters of positive comparable sales, with the company’s most recent quarter blowing revenue estimates out of the water by $100 million.
Underscoring the company’s prioritization of forward-thinking technologies, Domino’s is thinking years ahead, testing driverless cars in Miami with Ford.
On the digital side, the company’s implementation of automated phone orders via artificial intelligence assistant DOM should further grow its digital ordering business beyond the current 65%.
TSG President Charlotte Yarkoni describes how the group is changing the way Telstra is developing and managing its technology and providing the company with a mastery of software.
Quoting from this article how Telstra is using software: the telecommunications industry hasn’t been immune from these changes and part of the response at Telstra has been the establishment of a dedicated Telstra Software Group (TSG).
TSG business is segmented into three inter-related streams of work, all designed to deliver a more compelling customer experience
In closing, I believe every company must own and control their software destiny. Central outsourced dev houses are dead - development must happen with the business leading it, where business is part of the agile team, writing BDD test cases, and not requirements documents.