This article on the “Myths of Agile” was written By Dr. Justin Kelleher for Cora Systems and originally posted on MD&DI Online (Medical Device and Diagnostic Industry)
The Agile methodology helps to maximize the productivity of development teams by dividing projects into short iterations.
There are several misconceptions about the Agile project management methodology that need to be cleared up.
George Orwell once said, “Myths which are believed in tend to become true.” Of course, a story can become a myth, or indeed the myth often becomes a good story. This particular story has its genesis in the mountains of Utah, in February 2001, when a gathering of some of the greatest modern minds emerged with a set of 12 revolutionary software development principles under a proclamation called the Agile Manifesto. These evangelists declared that requirements should evolve through collaborative efforts using self-organized, cross-functional teams. They maintained that constant change, adaptive planning, faster delivery, continuous improvement, and evolutionary development were all, in fact, practices to be embraced. To traditionalists, this sounded like organized anarchy, in which control is handed from the executive team to what appears to be a group of socially inept developers; general mayhem and chaos would surely prevail. The Agile atheists were uncomfortable, and the war stories germinated. In this article, we will look at some of the guiding principles of Agile and debunk some of the myths that have flourished since the Agile Manifesto proclamation way back in 2001.
The Myths of Agile Debunked
The Agile methodology helps to maximize the productivity of development teams by dividing projects into short iterations.
One methodologist stated that “agile is not just a methodology, but a set of principles and philosophy.” The main principles of an Agile development methodology are as follows:
- Develop small (using “sprints”) while building and releasing the software to the client regularly.
- The requirements evolve constantly, but the timescale is fixed.
- Requirements should be captured at a high level, using wireframes or visuals where possible.
- Ensuring active user involvement is imperative.
- A collaborative and cooperative approach between all stakeholders is essential.
- There should be a focus on frequent delivery of the product.
- Testing is integrated throughout the project life cycle by testing early and often.
The enterprise software company VersionOne, in its 2016 10th Annual State of Agile report based on a survey of 3880 practitioners, clearly outlined that there is an increasing number of enterprises adopting Agile. Some 94% of all organizations surveyed by VersionOne now practice Agile. Approximately 24% of respondents to the latest research worked in organizations that have practised Agile for greater than five years, up from 19% in 2013.
The report states that
while agile adoption is increasing, there are still obstacles to overcome. The key barriers to further adoption usually hinge around culture, including the ability to change, general resistance to change, and management support. Interestingly, the majority of respondents pointed toward company culture as the reason for failed agile projects as well. Once these barriers are overcome, the limiting factor most often cited has been the availability of personnel with the necessary agile experience.
The top reasons for using Agile for the third year in a row are as accelerating product delivery (62%) and enhancing their ability to manage changing priorities (56%)—which is not a surprise as organizations respond to increasing market demands and customer expectations. By all indications, Agile is helping enterprises around the world succeed. For the past five years, the top three cited benefits of Agile were the ability to manage changing priorities (87%), team productivity (85%), and project visibility (84%).
On April 6, 1917, the U.S. Congress declared the “war to end all wars” on Germany. So what, you might ask, does this have to do with debunking the myths of Agile? What most people don’t know is that the United States came into World War I armed with a new secret weapon: the Gantt chart, developed by the all but forgotten project manager Henry Laurence Gantt, who was born in Calvert County, MD, in 1861. When Colonel John T. Thompson, the inventor of the Thompson sub-machine gun, received the Distinguished Service Medal in 1918, Thompson promptly sent a copy to Gantt with the following note: “A large share in this reward for the accomplishment of a great war task is due to H.L. Gantt and his assistants. The Gantt general control production chart was my compass.”
Gantt, a mechanical engineer and management consultant, had developed the charts just over 100 years ago. In the 1920s, he employed the charts on many infrastructure projects, including the Hoover Dam and Interstate highway system. A century later, while corporates attempt to move away from Gantt charts, projects still have interdependent tasks, and percentage complete must still be tracked and reported to completion. A project is still a project, a deliverable is still a deliverable, and, as such, project management principles still apply.
So, myth one of Agile debunked:
Agile does not mean you don’t project manage; Agile means you project manage constantly, to the very heartbeat of the development teams.
Keep your basic project management practices as your guiding principles, and stop thinking that the scale or complexity is new or unique. Henry Gantt certainly didn’t.
Another myth of Agile is that it is a “new” methodology. Man didn’t fly until 1903 and landed on the moon in 1969. Agile has been around since 2001, and it was simply the next generation in software development techniques that evolved from some incredible movements during the early Internet age of the 1990s. The methodology was heavily influenced by the Iterative and Incremental, Rapid Application Development, and eXtreme Programming (XP) movements. It was a “lift and shift” of certain practices from object orientation, emerging real-time modeling languages, code generators, and automated testing methodologies. Agile has been around for 16 years and like a good wine, it has matured well. This is good news for new adopters, as there is a multitude of successful case studies, literature, and expertise to leverage.
The earliest and most vocal anti-Agile outcries came from systems architects, who traditionally built conceptual models of the structure and behavior of the system through formal descriptions called “software architectures”. Experts warned in the early 2000’s that the architecture would become “brittle” with pervasive “spaghetti code” in the system because Agile did not prescribe formal design. The fear was understandable because as code is layered on top of older code with bigger user bases, it naturally becomes less malleable. For example, how many IT systems have you worked with whose architecture consists mostly of embedded “query calls”? Worryingly, the Agile Manifesto did not include—perhaps intentionally—the definition of a systems architecture. The reality is that in Agile, one needs more design; it’s just hidden inside the daily activities of the development teams. The design is inherent all the way through development, during every stand-up or sprint planning meeting. Today, the veracity is that all systems have an architecture, it just may not necessarily have formal architectural models.
So, who is responsible then for the overall architecture? The simple answer is that everyone on the team is responsible. However, if this is large-scale or complex distributed system, you’re going to need to invest some time architecting it. Do some up-front architectural design, then identify or hire someone into the role of “architecture owner”, or, in Agile vernacular, a “solution architect”. In regard to this myth, Agile does mean that developers improve the design of working code during the refactoring process, so make sure to add refactoring into your planning.
Another myth of Agile is that there won’t be any project documentation.
Before debunking this myth, let’s discuss why projects need documentation. Documents serve as a tool for collaboration across the organization by providing all the stakeholders with the necessary level of detail to communicate “up or down” on key project information. For example, stopping or course-correcting a project, (dis)continuing investment, describing the system, training or user documentation, and information on how to support the product. Oftentimes, documentation is produced just for the comfort factor, or the “we always did it this way” factor. The paying customer usually doesn’t see under the “hood” at the code base, so being able to touch and feel documents gives them the comfort that they are paying for something tangible. Of course, developers and technical personnel abhor doing documentation. So, at the start of every project, an agreement should be reached between all stakeholders on what documentation needs to be produced throughout the project. Documents take time and time is money, so documents carry a hidden cost. Therefore, documents should be sized and the cost agreed upon before being scheduled into a project.
In the past, requirement documentation was vast with a lot of repeated or unnecessary information within each requirement specification. In Agile, requirements are compiled as a collection of user stories that can be actively updated and maintained using product backlog management software. Increased collaboration throughout Agile development projects provides all stakeholders with a better understanding of the end product as it is being created, thus reducing the need for some specification documentation. There is redundant documentation and useful documentation. The trick is to find the right level of detail. Still, let’s say it like it is: Agile probably does mean less documentation. In Agile, you document only what has to be documented; the difference with other methodologies is the efficiency of the documentation.
There is a strange interpretation that there is little to no planning in Agile when nothing is farther from the truth. In Agile software development, work is confined to a regular, repeatable work cycle, known as a “sprint”. Usually, sprint cycles are two to three weeks in duration, during which time user stories are designed and built. At the start of each sprint, the entire team partakes in the sprint planning session. A “story” is a short description of a discrete piece of functionality. A story must include the usage scenario, the action taken, and the resulting behavior. Specific acceptance criteria, mockups, or wireframes can be added to provide additional details and guidance. The Agile product backlog is described as “user stories”, which is a prioritized features list, containing short descriptions of all functionality desired in the product. The main input to the product backlog will be the requirements represented as “epics”. An epic is essentially a large user story that can be broken down into a number of smaller stories. It may take several sprints to complete an epic.
It is important to note that an epic can span more than one project if multiple projects are included in the planning to which the epic belongs. Therefore, sprints enable prediction of feature completion based on the story and epic point estimations by the engineering team, the product priority, and the team velocity.
The vast majority of Agile frameworks involve frequent, regular, and evolutionary planning. If there are release plans, there will be a high-level agreement defining the product being delivered, the timescale, and the cost. However, the agreement and the plans are specifically set up to enable change, and there will be significant ongoing planning. Therefore, in Agile there is rigorous daily planning. The problem is not the planning; the problem is the competence of those carrying out the planning. It requires well-trained, specialized teams capable of self-management, communication, and decision-making. Invest in a specialist Agile planning coach for a period of time to mitigate this potential issue.
One of the biggest myths about Agile is that it is only to be used in software-based projects.
It is true that when the Agile Manifesto was published, it was in the context of software delivery, but Agile can be applied successfully in any business environment. Given the proven success of Agile development, an inevitable question for companies in the medtech, pharmaceutical, and other regulated industries is: Can we adopt Agile approaches in our environment? While medical device or pharmaceutical regulations do not prescribe a fixed life cycle model, activities are still presented in a sequential manner, which naturally drives organizations back into waterfall development. However, several medical device manufacturers have adopted Agile practices while keeping development in compliance with regulations, but conflicts still arise and decisions have to be taken in favor of agility or formality.
Agile team members have to master the technology and know the regulatory guidelines (for example the FDA regulations, security, or technical aspects). There are also Agile guidelines or standards available. In 2012, the Association for the Advancement of Medical Implementation (AAMI) released the TIR45, which guides medical device development companies on how to use Agile under stringent quality regulatory processes. TIR45 is an excellent source for neophytes or hardened Agile veterans alike to gain insight into using an agile framework in the medical device industry. It’s a great reference point and demonstrates a clear alignment with the all-important IEC 62304 standard. Undoubtedly, over the coming years, we will see the continued growth of Agile practices in non-software-based projects.
In conclusion, there is still a lot of conjecture and debate about Agile.
The harsh truth is that today, projects still fail. VersionOne states that the top causes of failure are a lack of experience with Agile methods (44%), a company culture/philosophy that is at odds with Agile (42%), and lack of management support (38%). The top barriers to success are the ability to change culture (44%), lack of personnel with Agile experience (35%), and organizational resistance to change (44%). One can conclude that the main problem with software development today is organizational culture and the ubiquitous problem of poor collaboration. But as long as you recognize this, and by keeping the above information in mind, your chances of getting better results with Agile will significantly improve.
An Agile guru recently stated that “agility within and of itself is a strategy.” It’s not a silver bullet; it’s simply a set of proven guiding principles to develop software faster. And don’t forget: “Fairy tales are myths, and myths are only myths because there’s a grain of truth in them.”
