How to integrate your AWS event-based system with external/legacy systems using RESTful APIs
It's one thing reading about the extreme DevOps practices of the unicorns (Google, Facebook, etc), and it's completely another thing trying to adopt DevOps in a legacy organisation. Thats why it's reassuring to see some real world experiences of DevOps practitioners doing it in legacy organisations, with plenty of cultural inertia to make it challenging. My previous book - DevOps for the Modern Enterprise, recounted the experience of Mirco during his time at Accenture, working with client implementing DevOps practices. This book - The DevOps Adoption Playbook, by Sanjeev Sharma, is the learnings of his time at IBM implementing DevOps at clients.
He uses the analogy of sports, and the plays that teams practice and master for certain situations, and uses this to derive DevOps plays that you can implement in the situations that need it. These then make up a playbook - team practices that you can use for implementing DevOps.
He identifies 3 core themes of all plays:
No matter how you arrange the entire organisation, the various teams, or even how individuals behave and act, it is a combination of people (teams), processes and automation that enables the true potential of DevOps to be achieved. Well organised and highly collaborative that are following inefficient processes, and are overburdened by rigid governance, don't have the right tools to enable and scale automation, or have legacy environments to deliver the application to will fail. True transformation - and that is what a DevOps adoption is, a transformation - requires transformation of all three people, processes and tools. None comes first. They cannot be adopted serially, and none can be. ignored. Teams are the focal point of the transformation. Processes guide teams on how to do the work. Tools allow the the processes to be repeatable, scaleable and error-free.
- Minimising cycle time - not only for dev-test and deployment, but also for data-center latency, project approval, can approval, financial approval, acquisition approval, management approval
- Reducing Batch size
- Establishing the right culture - one of my favourite sections. He goes into details about DevOps is first a cultural movement, and talks about Conways Law. Individuals can be the eventual enablers, or the ultimate bottlenecks. Only individuals with the right intentions and more importantly with the willingness to to change can make the necessary changes to cause transformation. Ultimately, even if the teams are well organised to enable trust, collaboration, and communication - if one or more individuals choose not to participate and not to overcome the cultural inertia, then change will not happen. If you look at culture as purely being how people behave and interact, then making sure you have the right people on the team is eventually the most important measure and cause of right culture. Probably the most extreme example of steps companies take ensure they have the right people on their teams come from Amazon - where they pay people who are unhappy to leave. They would rather pay people to quit then keep them on the team, hurting it the company culture.
He then identifies these main plays:
- Establishing Metrics and KPIs
- Adoption Adoption
- Integrated Delivery Pipeline
- Continuous Integration
- Continuous Delivery
- Shift Left: Testing
- Shift Left: Ops Engagement
- Continuous Monitoring and Feedback
- Release Management
Certain specialised plays like DevOps for Mobile, Mainframe, IoT and Big Date are mentioned as well. In the next sections, we will review some of the other plays.
The book has multiples sections dedicated to Value Stream Mapping as a way to identify waste.
DevOps for Innovation
In Chapter 5 - DevOps for Innovation - Sanjeev talks about "optimising to innovate". When a legacy enterprise wants to innovate - either a completely new service, new business model, new platform, or even a new market - this now thing will be dependant on something from the legacy platforms. So this will hamper the speed at which the new innovation can be delivered, because the legacy systems will be slow to make changes to cater for the innovation. You thus need to optimise the legacy, by using DevOps, before you can expect to innovate.
Play: DevOps Platform
In Chapter 5, he starts with a discussion on Cloud, and the different Cloud consumption models. Beyond just Public vs Private Cloud, he mentions other models like Dedicated, Local and self-managed vs vendor-managed Cloud models. This leads to a discussion around the importance of a DevOps Platform (PaaS), which allows you to adopt at a very low cost of entry, because you dont need to craft a delivery pipeline and toolchain. The main services that a DevOps platform provides are:
- Source Management (git, etc)
- Deployment automation
- Middleware configuration
- Automated testing
He ends off with a discussion on containers:
the unique application code inside the container is isolated from the environment outside the container. It is almost a DevOps nirvana. The developers do need to be concerned with the environment outside the container, and the Ops team do not need to worry about the code, config and dependencies inside the container
Of the core themes identified ealier, delivering small batches was about multiple teams using short sprints that deliver code in small batches. But if these need to be integrated and built into a single deployable asset, then we have defeated the purpose, and broken another core theme of reducing cycle times, because we know have to wait for external teams. The core issue behind this is usually a monolithic architecture, that forces big, co-ordinated deploys. However, a microservices architecture will allow you to deploy each module independently, preserving the core themes of small batches and reducing cycle times.
Cloud Native apps are the culmination of DevOps, Microservices and Containers. 12-Factor apps is a methodology on how to design and develop these microservices.
Cloud-native applications, by definition, require a DevOps Platform (PaaS or CaaS) in order to deploy and run. They are developed and delivered using a 12-Factor App methodology. Small batches, rapid delivery, scalability, and anti-fragility are some of their core themes.
This leads on to the importance of APIs and Plays for building an API economy.
Getting DevOps working in individual teams is one thing, doing across the organisation is completely different. In order to scale DevOps, he identifies various plays for scaling.
Play: Building a DevOps CoC
DevOps Center of Competency can be a source of DevOps expertise, enablement, tooling guidance, as well as DevOps coaches who can help teams adopt DevOps. DevOps coaches are modelled after the Agile coach - their job is to work themselves out of a job by leaving a team self-reliant that can deliver efficiently leveraging DevOps practices.
Play: Team models
He references the classical Spotify model of Squads, Tribes, Chapters and Guilds as a way to scale DevOps, share skills and knowledge, while ensure teams are self-directing.
Play: Tool standardisation
As a follow-on to the above team models, in order to have fungible people, visibility and traceability, you need to standardise on tools.
He suggests a balance between 2 extremes:
- tool sprawl: every single team using a unique tool, which prevents visibility
- unreasonably standardising on a single tool set: you cannot switch all apps no Node.js, or move them all to the mainframe
So the balance is a limited choice of tools that are well integrated, thus delivering consistency with choice. He gives an example of IBM Bluemix's open toolchain:
DevOps Transformation Anti-patterns
- The "DevOps" project: Adopting DevOps is not something that is done once and then done with.
- Lack of ownership: You cannot bring about change without clear ownership and responsibility of who is responsible for changing what, with what resources, and by when
- DevOps adoption in islands: DevOps adoption starts with pilot projects, in order to learn. A common anti-pattern is when executives keep this isolated to their domain
- Change of reporting structures, new leadership roles, new silos and and DevOps teams: Naming someone VP of DevOps and creating yet another silo (DevOps is meant to break silos)
- Outsourcing DevOps: While contractors and experts are vital to bring in new skills, the ownership cannot be outsourced. If the team does not see their own execs taking ownership, there is less motivation to overcome cultural inertia
- Contracts: Rigid contracts that force vendors to communicate and act in a certain way
As an aside, if you just want a quick guide on how to do a DevOps transformation, then check out the free Supplemental chapter from the book Accelerate. In really is a simplified version of Sanjeev's book.