Flask on AWS Serverless: A learning journey - Part 2
About 3 years ago I learnt some basic Python, which I've used almost exclusively to
I recently had such a huge Aha moment, my own Eureka - I literally heard the light bulb click in my head, I tilted my head sideways, and then I understood it all. It explained the weird moments I had earlier in my career working with my older and wiser teammates, and it especially shed light on my recent experiences with management, and my career.
Let's step back first. In my (short, about 15 year) career thus far, I was usually the youngest, and the most excitable person in the team. I was eager to jump onto new projects, and change the world. And I usually did. I would introduce new platforms, technologies, or just new processes and ways of work that changed, challenged or even discarded the "way we always did it". And it just came naturally - I never tried to be different, but I would look at something, and wonder if we couldn't do it easier or faster. And most of the times it was something dead simple and obvious - and I would wonder why the more experienced, smarter guys didn't think of it. When I would tell them about, and introduce the idea, they usually started off patiently explaining to me why "thats the way we have always done it", then when they see me not convinced, they would give me that look. Its hard to to explain - it's a stare - a long, far away look, that would have perhaps a glint, then just turn blank, into the abyss. I got used to it - they stuck in their old ways, I know better. Not all the time, but sometimes, I was right.
As I progressed in my career, I took on new challenges. I changed jobs, and developed and launched new systems and products, that was new and exciting. Another new job brought me to run a DevOps team in a large, lumbering telco, stuck in their old ways. Perfect hunting ground for me. Our team was different, we automated everything, we were multi-functional, full-stack guys. We broke all the rules - CAB, ARB, you name it. We made a name for ourselves. People recognised us for doing the hard stuff, and getting it right. But I would still often see that look, glinting, staring at me.
But then I started getting bogged down. The governance wore me down. I was tired of having to break the rules all the time to get stuff done. I sometimes took the easy way out, the path of less resistance, and abided by some of the processes. My team got frustrated - "why arn't we going ahead, we losing time?". I replied with "we have to choose our battles - lose the battles, win the war". Maybe they believed it.....maybe I believed it?
Another job - and yet more fertile hunting ground. This time - throwing out an old system and archaic processes, and putting in all the bells and whistles: in-sourcing development, running in the cloud, APIs, feature teams, agile culture, DevOps, new ways to procure, platforms not products - the whole shebang. Delivering real value, literally changing the world. I had proved that I could deliver, and be part, and the impetus for a radical transformation.
At first it started with escalations from management to accept late, poor quality user stories into the sprint. Demanding mid-sprint that we work on things that were not in the sprint backlog. Then we were forced to deploy to non-cloud environments, impacting the ability to automate and use the pipeline. Quality started dropping. The burn-down charts showed the decreased velocity. I clearly showed, then begged management to see the impact it was having on the team, on delivering value. But its hard to make software architecture matter to them, when they couldn't see the cost of poor quality. Executives placated me, saying I need to be patient. They used the old worn-out lines on me: "that change is hard in this company" and "choose our battles". I countered with the change required for Digital Transformation - to flat structures, feature teams with the authority to map their own path, less governance, more power to the people. They were excited, but realised the leap was just too great!
Quality and team velocity kept on dropping.
And...the corporate antics wore me down again. It became harder and harder to do my thing, and get stuff done. The career pendulum swinging again - or is it more? I started to let stuff go. I stopped hustling to get stuff done. I sighed, resigned myself, and took the path of less resistance - I buckled and let the governance win. People came to me, and I whinged about how hard it is. And then I looked it the mirror, and saw that look. That distant, blank look.
And thats when the Aha moment hit. Lit a slap in the face. Its when I realised: they were all once just like me. They - my previous colleagues, who always said "this is the way we always did it" - were once young, and ready to change the world. And they probably did, for a while. Until the system got them, bore them down, sucked them dry, and left them like empty shells. Coming to work, doing a 8-to-5, and leaving. With no passion left any more for innovating and improving, but rather just maintaining the status quo.
Resistance is futile: I now have that same look.
Some of you may have recognised some of this as a resultant of behaviour called Stalled Agile (amongst other Fake Agile activities)
We see many examples where Agile becomes an increasingly important dynamic of the software development group but ultimately fails to win support from the top management. Initially, the fact that software development is operating in an Agile fashion, while the rest of the organization is functioning much more slowly and inflexibly in a traditional bureaucratic fashion, isn’t too much of problem: the management is happy to see the faster production of software.
But as the agile movement spreads, the tension between the two different modes of operation can start to become a source of tension. The software developers become frustrated with the slow pace with which the rest of the organization uses its software and start lobbying for change.
At some point, the lobbying may irritate the top management as the software developers challenge well-established management practices. As a result, the management may close down Agile implementation and dismiss the Agile leaders and coaches. Of course, the top management doesn’t use those words. Instead, they declare victory: “We are already agile. We don’t need agile leaders or coaches and more.” As a result, the Agile journey can stall or die, or at least go underground.
Regarding the conflict of only IT going agile, and not the rest of the business:
The hope of the founders of the Agile movement back in 2001 was, I think, that if they could get Agile teams going in IT, the teams would write great software at lower cost, and then the top management would leave them alone, and the whole firm would become agile. It turned out not to be the case. Over time, the friction between the agile and non-agile parts of the organization grows. The result over the last ten years or so we have seen that either Agile takes over the whole firm or the bureaucracy crushes Agile.
What we have learned is that these are really two different philosophies, values, goals, attitudes and mindsets. So, there are inevitable conflicts. Sometimes the tensions fester for years. Often, the end result is that Agile is closed down. For instance, we saw that a couple of years ago at Intel, and it has just happened at General Electric. But of course, the reasons why Agile was introduced in the first place still remain, and may be even more pressing. Thus, suppressing the Agile movement within a firm isn’t the end of the story. Sooner, rather the later, the firm has to consider again, "How do we become more flexible, more agile?" That question doesn’t go away.
The problem is widespread, even in organizations that are actively embracing Agile at the team level. In surveys that we conducted at Scrum Alliance, we found that some 80-90% of Agile teams perceived tension between the way the Agile team is run and the way the whole organization is run. In half of those cases, the tension was rated as "serious."
While most large organizations are still learning how to master operational Agility, the main financial benefits from Agile management will flow from the next Agile frontier: achieving strategic Agility.
Fake agile is too concerned with "branding agile" - running it like a project, unti its done. Rather, you need to take an agile approach to going agile, like Microsoft, which started with one team, then extended to others.
There is some hope though. Executive coach Mark Holtshousen explains in his article Too Tired to Lead: Curating Energy and Resilience, that when we were younger, we started off with "wild abandon", putting in lots of energy, which led to success, and which later would become the point of reference to which we would compare all future activities against. But as we become older, and wiser, we put in less energy, which leads to less success, and this is the kicker - we then start critiquing, and comparing this lack of success to our earlier successes, when ironically, we had put in more energy in order to make it successful.
We move from participation, to observation, to critiquing.
Mark claims this lack of energy comes not from a lack of motivation, but a lack of physical energy. In my case here though, I am not quite sure. I dont think I lack the physical energy, because as he later says in the article that we need to sleep, eat and exercise, which I purposefully make sure I do. I think there is a lack of motivation that originates from a lack of Transformational Leaders in my structure, specifically around this point:
who feel supported by their employers, who have the tools and resources to do their work, and who feel their judgment is valued, turn out better work.