Fads & failure: What went wrong with TQM, Lean and Agile?

None of the big management movements of the last decades delivered on its promises. Losing faith in transformation would still be a mistake

by Niels Pflaeging

Over the past four decades, several global trends, or movements have provided promising, constructive impulses for organizations and the world of work. The latest of those trends became known as Agile. Before that, equally bordering on hypes in their time, came TQM/Quality and Lean. Although these approaches pointed in the right direction (at least initially), they ended up failing to fulfill their potential. In this article, I argue that there has been a common, big blind spot in all these awe-inspiring movements. This blind spot has been their failure to offer suitable approaches to organizational development and transformational change. The crux: Without proper insight into transformation (and willingness to act accordingly), Agile & Co. were doomed to work like dextrose pills: Delivering short bursts of energy – followed by a swift return to the prior state of command-and-control.

Command-and-control systems aren’t easy to overcome. Interventions within a systems often produce observable short-term effects. In isolation, however, individual interventions will never have an impact on a system as a whole that deserves to be called transformational. If no or too few impulses capable of "unhinging" the system are provided, then lasting or profound transformation of value creation cannot and will not take place. There will be no long-term impact of effectiveness, productivity, or results. In other words: In the long term, stuff like trainings, certifications, local experiments,, implementation of tools, team interventions or individual/group coaching can never replace working on the system.

Being half right is not enough

Agile methods like Scrum came along without an implementation approach, as Gunter Verheyen documents in his brief history of Scrum-as-a-fad. From the beginning, the focus of Scrum in particular was on certification and training, reports Verheyen. Which is fine, as long as one does not expect certification and training of people to transform organizations. There is a difference between training people and acting upon a system. Similar things can be said about Lean and Total Quality Management: Their business models were also based on trainings and certifications, by and large. Related fads were especially effective as qualification businesses, while there was an assumption between service providers and clients that such qualifications would “somehow” lead to impact. Most corporate initiatives today echo this dogma: A training delivered means the job of change, or even “the business of transformation” is done.

Let's take another look at the history of the TQM, Lean, and Agile movements. The three trends have in common that they received a big boost from resonating well beyond their original niches, after a while, which were industrial production and software development, respectively. At the same time, the movements also industrialized themselves. Which means: More and more quality people, leaners and agilists began to make a living from related professional activities – but without ever working on actual transformation of their client's or their own organization's systems. Instead, certification, trainings, permanent small group support, measurement, technical-analytical tools and software tools became the dominant business models within these movements.

An example: Between 1991 and 2011, Lean became more and more closely associated with Six Sigma. At some point, Six Sigma even overtook Lean in popularity and market power, as Six Sigma became a formidably lucrative, certifications and trainings business that got people jobs and promotions. Six Sigma, however, was never much than that: A recruiting market fad. While Six Sigma strived as a qualification business, it became clear that the Lean/Six Sigma industrial complex had ceased to become a force for real-world change and company impact. People were hoarding certificates, yes, but hardly any business impact could be expected by companies showering money upon Lean initiatives. This worked well for a few years for all parties involved. Then Lean perished. Now think about this story again, replacing the word Lean with Agile and Six Sigma with Scrum - and you will notice that the story rings just as true. One might say that history repeated itself with Agile and Scrum, about 15 years after the implosion of Lean.

The problem: Without proper transformation of entire organizations, neither true quality, nor lean or agile systems can emerge. In effect, all three approaches would have required substantially higher degrees of decentralization, functional integration and team-based self-organization than what is common in today's organizations. Which of course is precisely what the BetaCodex movement has been advocating since 2008.

Back to TQM, Lean and Agile. Over time, the actors within those movements found themselves in a dilemma. On one hand, the surge of business and income generated from the respective trends created the image of successful, vital movements with numerous, economically successful actors and members. On the other hand, these actors found it harder and harder to deliver on the expectations, short-term and long-term, that were created by business media and within organizations willing to pour millions of Euros or Dollars into the stuff. Without pulling off successful, profound transformations in practice, people spending money on the fads were condemned to produce very limited, short-term boosts, followed by immediate set-backs.

Promoters of organizational renaissance are in a pickle

At the start of my career as a researcher and consultant, I had to painfully experience that kind of dilemma myself. In 2003, at the age of 32, I officially joined the Beyond Budgeting Round Table as a director, thus becoming the 5th core group member of a movement in its own right that had been founded back in 1998. At the time I signed up as a full-time director, the Beyond Budgeting model (now: The BetaCodex) had already been fleshed out conceptually: Our group had largely settled on a well-defined set of 12 principles; there were more than 20 carefully documented case studies grounding the model; we had a pretty solid community of clients/organizations that supported our research – ideally as well as financially. At that time, we already knew how Beyond Budgeting worked as a model and what it was all about. The only thing that was missing were those "Beyond Budgeting transformations", in organizational practice. In other words, we were lacking cases of organizations that had adopted the Beyond Budgeting model through our community's own efforts. Even during the 7th, 8th and 9th year of our movement, there was no such transformation case of ours to show for – despite the formidable commitment of the directors, and in spite of the lively interest within our community. So the clock was ticking on us. It was now 2007.

To some within the group of research directors of which I was part (we there were a group of eight by 2007, with a few additional and capable individuals showing interesting i sign up), the situation was causing me and my mentor, Robin Fraser, a great deal of stomach ache already. But others within our team weren’t bothered at all. Most of us were doing pretty well, economically – thanks to income from public speaking and trainings, as well income generated through membership fees from the paying community. Economically, we were in a pretty neat position, back in 2007, and we could all notice growing public interest in our work. So things were looking good, on the surface. But there was more good news: We all had options how we could further exploit and scale our individual and collective businesses related to public speaking, trainings and community work, or perhaps to develop the occasional software tool or physical product. Diagnostics, forecasting tools and CFO advisory were obvious business options for us to exploit, with Beyond Budgeting as the platform. Things were good.

In the eyes of those among us who had their stomachs aching, the alternative to milking the cow seemed somewhat harder: We needed to master the art of transformation of real-world organizations, large or small, and get companies and not-for-profits to adopt Beyond Budgeting, in earnest. We had to become consultants in the way that some of our historical predecessors had attempted full-fledged organizational transformation. Among them were organizational pioneers (and, well, near-geniuses) like Mary P. Follett, Kurt Lewin, Eric Trist, and Douglas McGregor. They had all gotten into change work with corporate clients. McGregor had probably gotten closes to conducting transformational change with clients. This path, however, was going to be an arduous journey, that much was clear. Because some among our group (me included) were already experimenting with the Leading Change approach that had recently been developed by John Kotter. Our attempts at applying Kotters' approach had produced mixed results. For us, in the year 2007, a move towards promoting consistent transformation would have had to go hand in hand with changing the Beyond Budgeting brand, with adopting a different public discourse and modifying our combined business model as Beyond Budgeting “directors”. As it turned out, the larger part of our director's group would not opt into this challenge.

The state of the Agile movement, today

A similar development has been taking place in Agile, for several years now. In the 2010s, Agile turned into a billion-dollar business that also became highly industrialized. Despite claims to the contrary, however, the Agile movement is not producing transformation worthy of the name, neither on corporate levels nor on a divisional or departmental level. That should not come as a surprise. Because within what might be described as the Agile Industrial Complex, the dominant business models and sources of income for agilists are the following:

• Certification/trainings with a myriad of titles, qualifications, and badges on offer.

• Continuous team support, or, more negatively framed, baby-sitting – often referred to under titles such as Agile Coaching.

• Software tools supporting Agile teams and Agile work – from providers such as Atlassian.

• Technocratic tools and the associated large-scale implementation projects, especially those associated with Agile Scaling that employ "implementation frameworks" such as SAFe, or performance systems such as OKRs.

None of these business models is suited to produce actual Agile transformation (which would deserve the name). Good intentions, or appealing to purpose and to people's "Agile mindset" do not change that fact.

Despite claims to the contrary, the agile movement is not producing any corporate or even divisional transformations worthy of the name, anywhere in the world, nor in any kind of industry. That should not come as a surprise.

My point here is not that Agilists have the wrong motives. Much the contrary: I am convinced that pretty much all Agilists have the best of intentions. I believe that many, if not all people who promote Agile believe they are making contributions to the world's great journey to Agility. A few years ago, people seriously thought that bringing Scrum into schools would change the world, as they became convinced that "agile" approaches were badly needed in education. All of this, combined, leads me to believe that there was never a shortage of missionary zeal within the agile movement. We want to do good, right? But, as W. Edwards Deming said:

Good intentions are not enough. Everybody already has the best intentions!

The problem: Existing systems, such as companies, not-for-profit organizations or the education system cannot be changed through certifications, trainings, coaching, tools or continued support to value-creating units/teams (external or internal). If you limit yourself to such services or products (which can make good business sense, at least in the short-term), then you will always remain in the mode of symptom caretaking. The underlying problems or messes, will never be resolved, though.

Furthermore: When symptom caretaking services are industrialized further, and offered on a large scale, then the movement's economic expansion will eventually force its members to produce ever-increasing claims of effectiveness. This will produce some sort of faddish bubble, eventually. Worryingly, precisely such an effect has been noticeable in the Agile scene, for at least ten years now. Since 2023 or so, it is clear at least in some countries that the Agile bubble has burst. Similar bubble formation occurred in the Lean movement during the 1990s. It was followed by dramatic, abrupt decline, which might also be described as an implosion of Lean business and reputation.

Working in the system will never bring about a system's transformation

Among the lessons that my work with the Beyond Budgeting Round Table (and the many wins and defeats associated with that work) taught me, is this:

There is a hell of a difference between working in a system and working on a system!

Working in the system is vital, of course. No value or business performance can be produced without the daily business, and without improving it - kaizen-style, ideally. At the same time, that type of work needs to be accompanied by constantly taking the design of the system itself into view. A way of grasping the difference between the two types of work is this:

• Work IN the system, a.k.a. System Optimization works by straightforward selling (or adopting) of services or products to a client.

• Work ON the system, a.ka. Overcoming the System, on the other hand, requires winning a mandate, or authorization first.

The latter may appear somewhat more difficult, or tedious, at least in the short term. But pioneers like Kurt Lewin (1890-1947) already understood that it may be more effective and more fruitful, in the long term, to change an entire organizational system, compared with attempting to change individual people, one by one, within the context of existing systemic environments. Contemporary, invitation-based approaches to transformation, such as OpenSpace Beta, take this fundamental insight about the nature of organizations and people at work into account.

There is a hell of a difference between working IN a system, on the one hand,
and working ON a system, on the other.

Without approaches suitable for transforming entire systems, within acceptably short periods of time, we will forever be condemned to scratching the surface of command-and-control systems. We won’t produce profound quality-orientation, nor truly lean organizations, nor mind-blowing agility. The reason: Most organizations around the world are still stuck in command-and-control mode. The restrictions that such systems impose on people and interactions need to be lifted, if we want to release the full potential of concepts like Lean, Agile, self-organization and organizational democracy.

This has consequences for reformers inside and outside of the organizations that seek transformation. Within organizations, promoters of change need to make the case for working the system – instead of sidelining with short-term optimization efforts. External "reformers" and consultants, on the other hand, should examine their own business models: Does your business model aim at the transformation of entire systems of organizations, or are you merely promoting tools and client-sitting? If you want to see transformation occur within your clients, then most of your paid work should aim at overcoming the client’s systems – directly or indirectly. Regardless if that paid work comes in the shape of a workshop or a concept session, a learning tool, a product or a coaching gig.

Organizational transformation doesn’t happen by chance. And there’s nothing miraculous about it either. It’s just damn hard work.

---

For more on very fast organizational transformation that deserves the name, visit the OpenSpace Beta web page

For more about the shortcomings of Scrum and Agile, and how to overcome those shortcomings with time-orientation, read the article by Niels Pflaeging | Red42 about the fallacy of Sprints.

For more about the history of Lean and how to fulfill upon its promise with time-orientation read the article Did Lean fail?

Also read the articles on the Red42 Substack Transforming organizations for good. Fast