I want to try and describe a topic I have been confronted with lately related to agile project delivery – discipline.
There are a number of ways projects and programs can fail. Teams that aren’t 
equipped with engaged and talented professionals, suffocating organizational politics, or teams with limited/no access to business decision-makers can all drive the probability of project success to near zero. And these are just a few of countless anti-patterns. Agile and Lean frameworks have emerged in the last 15 years to address some of the common anti patterns, and for many teams, this helps. Unfortunately, there are also some incredibly common misconceptions in organizations about what “agility” means. It’s expressed in a number of ways and many can be boiled down to one incredibly common phrase, “agile means winging it”. Of course this is hogwash and, without guidance, many new agile practitioners adopt this world view.
In fact, this is one of the most common misconceptions about agility and the process to achieve self-managed, high performing development teams.
First, give the teams some rudimentary training activities, followed by a “let teams figure it out” strategy. Just stick teams in a team room, give them daily standups and the 3 questions, sit back and watch the magic happen. After all, these teams are supposed to be “self-managed”, right? Doesn’t that imply that we stop managing?
This line of thinking is misguided, dangerous, and will ultimately raise the probability of failure to new heights. It might even be worse than the waterfall (command and control) philosophy of the last 30 years. But, if we dig just a little bit, we quickly uncover in the origins of agility, through methodologies like extreme programming (XP) and Scrum, that exactly the opposite is true. The rigor does not go away, but the philosophy changes drastically in a number of key ways.
Kent Beck, the father of extreme programming, test driven development and one of the original signatories of the agile manifesto said in his early book,
“When I first articulated XP, I had the mental image of knobs on a control board. Each knob was a practice that from experience I knew worked well. I would turn all the knobs up to 10 and see what happened. I was a little surprised to find that the whole package of practices was stable, predictable, and flexible.”
Does the phrase “Turn all the knobs on the control board to 10” sound like “winging it”?
Discipline and Rigor in an Agile Software Project
If all of this sounds like theoretical pablum, then let’s get concrete. Below is a list of activities in a typical SDLC, followed by some practices we have found that work, and how we might “turn the knob to 10”. To be clear about this list, some of these activities would be seen in projects of old, just not executed continuously. Agile constitutes a mind-shift, similar to how lean manufacturing shifted thinking about building physical products. For example, in 1970, American manufacturers couldn’t believe that building large consumer products “to order” was reasonable. Big batches, queued up at every step in the value stream was viewed as “efficient”. In 2014, the philosophy has changed drastically.
(This list is not meant to be exhaustive, but simply examples to illustrate agile activities)
- Some ways Agile teams amplify Project Management
- Continuous Planning: Shifting the focus from a one-time uber-planning event, towards constant planning and re-planning. This helps teams rely less on “the plan” that was developed in the beginning of the project (when we know the least), to relying on regular planning activities (as we learn).
- Small Releases: Planning and committing in small releases, versus big bang delivery at the end. This step is challenging as the work to make this happen forces leaders to challenge the “big batch & queue” models from the manufacturing processes in the early to mid 1900’s. (interestingly, manufacturing teams figured out this fallacy as early as the 1940’s and 50’s through the work of visionaries like Taichi Ohno)
- Customer Engagement: Ensure the customer is part of the team, or always available to the team to shrink feedback loops and allow for immediate decision making.
- Process Tools: Use tools like the planning game, the system metaphor, blitz planning, walking skeleton, MVP, and planning poker to ensure constant and realistic planning is short bursts
- ROM Estimating and Error Bars: A project managers ability to build trust by constantly delivering is important. One risk to expectation setting is a project managers ability (and fortitude) in knowing how to be predictable using rough order of magnitude estimating and clearly articulating the delivery date “error bars” at the end of a release or project. Even when a date is articulated and credible (meaning a date is imposed for legitimate reasons), the discipline of setting appropriate expectations by de-scoping is critical.
- Hyper-Prioritization: Helping teams pull critical features to the top of the priority stack continuously. Manage risk in this way so that we avoid ignoring issues until the end of a project.
- Team Effectiveness & Psychology: Constant evaluation of team effectiveness and health. If teams come to a standup daily and talk about missed tasks, or failure to integrate code, or failure to interface with customers, the project manager has the obligation to stop the line and help fix it. Often, even when the question “what is impeding your progress” is asked daily, most people won’t tell you there’s a problem. This is where the term “spidey sense” actually helps. But it only helps when you know how listen, and evaluate reality against a set of agreed upon practices by the team. That means that we need to expect the team to actually do this stuff religiously.
- Vision, Roadmap & Release Plan: Ensuring we have a vision for this project that it’s well articulated and that we have a consumable roadmap of prioritize features/stories. This plan is constantly re-evaluated and compared against reality.
- Risk Management: Activities such as evaluating the interface between vendors and shared service groups within the organization. If team cannot get a shared service in the room to commit with the team and work with the team in a collocated fashion, they have just inherited a project risk and dependency activity.
- Visual Control: This can come in many forms, but one well known technique in IT is the Kanban Board. Slightly different than manufacturing Kanban, the basic premise is that we need to visualize the teams work, limit their work in progress (WIP), and focus our teams on “smooth flow” through the IT system. The biggest killer of effective delivery in a complex system are invisible queues (places where work products sits in a “wait state”). If you visualize the steps that work items take to get through a system, you immediately recognize the issue. You will immediately see very few value adding activities awash in a sea of waste. Most work, when allowed to be invisible, is “sitting and waiting”. And, the more you allow teams to multi-task, the more waste you are allowing in the system.
- Some ways Agile teams amplify Business Analysis
- Continuous Requirement Elicitation: Agile teams amplify requirement elicitation. We do this by forcing ourselves to understand the businesses view of the requirement using user stories, pictures, models, wire frames delivered by the real users of the request. Different than a BRD, stories allow us to understand why the customer wants with they want, and also allows us to create domain specific acceptance criteria through collaboration with them.
- Triad Development and Collaboration: BA staff are constantly interacting with the QA and developers and evaluating the work product as its produced. This requires enormous amounts of collaboration and work, but the output is typically grounded in reality, versus the fiction of “the plan”.
- JIT Requirements: Rework is lessened as BA’s stop building huge requirement inventories that age and rot. There is a constant questioning and “de-scoping” that happens throughout the projects as requirements are built as they are understood. This allows the team to “maximize what we don’t build”.
- Some ways Agile teams amplify Software Development
- Pair Programming: This technique allows code to be written by a pair, where the “driver” writes code and the “observer” is freed to review the code being built real time, as well as help keep the strategic intent of the feature in mind while code is written. This technique tends to build high quality, well-structured code, fast.
- Test Driven Development: Write a test, watch it fail, write code to make the failing test pass. TDD helps dev teams write incredibly focused, well designed code that is also “change tolerant”. This is incredibly important as we introduce refactoring. Instead of eliminating change, we relish and encourage it. We also employ the same concept using BDD, or behavior driven development where requirements are evaluated using code.
- Refactoring: Mentioned above. This is critical as we want to allow business users to change their minds frequently so we build exactly the right thing. This takes extreme discipline.
- Continuous Build: One dirty little secret of system development projects is that, no matter if the code builds on your machine (or in your development environment) it is NOT done. In fact, many, many projects fail because they are never able to integrate and deploy the code. This technique allows us to get over this hurdle early, and forces teams to confront reality very early in their project lifecycle.
- Other Concepts: Agile teams amplify software development by focusing on code craftsmanship, standards, pattern harvesting, and evolutionary design techniques as well as others to ensure that we are building quality into our products versus checking for quality at the end of the project.
- Automated Testing: Automated unit testing, pairing, TDD, ATDD, BDD, daily check-in, continuous integration, heavy interaction with BA/QA/Ops, etc are completely necessary
- Some ways Agile teams amplify QA
- Triad Development and Collaboration: Agile teams amplify QA by forcing QA specialists into collaborative discussions (daily) on individual user stories and features. Often you will see QA people working on acceptance criteria and helping to train others. With a testing mindset in collaborative discussions, many features are built with the end user in mind, versus without context.
- Automation: QA also is responsible to set up automated testing mechanisms for the “simple stuff” so that we can have the humans move onto the complicated requirements. This requires QA to learn new techniques, become more technical, and interface often with the team (business partners included).
- Full Team Members: The QA staff are now full-fledged members of our teams instead of a department that gets handed a work product to evaluate at the end. This is a large shift for this group and one that forces them to be incredibly rigorous in their delivery.
- Some ways Agile teams amplify Coaching
- Leadership & Psychology: Agile teams amplify leadership by creating a social contract between professionally equipped teams and the goals and outcomes they are to achieve.
- Collaboration: The command-and-control management style tends to build teams that only know how to execute tasks and rarely improve. This leadership model, however, is bar none the easiest to implement, and typically yields shitty results and demoralized teams. We are trying to get our teams to recognize the business need / business value and the rigor necessary to execute and to improve continually while they deliver.
- Value Stream Management: The value stream manager role is a servant leader position to support the team when they can’t deliver in (often) incredibly challenging environments.
- Gemba: Translates to “Go and See” in Japanese, this technique is critical for leadership, and one that tends to be lost in many organizations. This concept is important as every team has different needs and contexts. There are many developmental levels as well. For instance if a team needs to deliver a feature using technology that is new to them, a VSM will take a more hands on approach to teaching / guiding. If teams are well equipped for a challenge, a VSM might employ a more Socratic style to effect appropriate change and challenge the team to improve. The only way a VSM can know what to do and how to help is to GO AND SEE.
- Some ways Agile teams amplify Operations
- DevOps: Continuous Integration and Delivery: Agile teams amplify the operations and deployment of software by forcing teams to integrate every single day through automated “hands off the keyboard” techniques. Continuous deployment is delivered through a four tiered environment structure so that we can show customers real, legitimately working software every single day (Or as fast as we can deliver it) That’s important because it allows us to shorten feedback loops to incredibly small proportions and remove waste from the system. For instance I can write the feature today, check it in, build it, have all the tests pass, deploy it, and show it to a customer and get feedback in the same day. This avoids the shit storm of integration issues most large projects suffer
- TQM: From an IT standpoint, the total quality movement means something very similar than the manufacturing technique of the last 30 years. Essentially the team is committed to build quality into our systems versus checking for it at the end. Buggy, poorly designed, rarely tested mountains of crap software are regular fixtures for Operations teams. Using many of the agile techniques listed, the number and type of defects introduced into a production state is deeply reduced. They are also delivered into production in tiny, easily understandable packages that are easy to remove. The net effect is an incredibly stable environment that has reduced the risk of change to such a degree that, in many cases, the deployments can be fully automated.
- Full Team Members: The Ops staff are now full-fledged members of our teams instead of a department that gets handed a work product to deploy at the end. This is a large shift for this group and one that forces them to be incredibly rigorous in their delivery, as well as challenge designs of the team early and often. This allows the team responsible for operating the system to help affect it’s design.