I want to try and describe a topic I have been confronted with lately related to agile project delivery – discipline.

There are a number of ways projects and programs can fail.  Teams that aren’t equipped with engaged and talented professionals, suffocating organizational politics, or teams with limited/no access to business decision-makers can all drive the probability of project success to near zero.  And these are just a few of countless anti-patterns.  Agile and Lean frameworks have emerged in the last 15 years to address some of the common anti patterns, and for many teams, this helps.  Unfortunately, there are also some incredibly common misconceptions in organizations about what “agility” means.  It’s expressed in a number of ways and many can be boiled down to one incredibly common phrase, “agile means winging it”.  Of course this is hogwash and, without guidance, many new agile practitioners adopt this world view.

In fact, this is one of the most common misconceptions about agility and the process to achieve self-managed, high performing development teams.

First, give the teams some rudimentary training activities, followed by a “let teams figure it out” strategy.  Just stick teams in a team room, give them daily standups and the 3 questions, sit back and watch the magic happen.  After all, these teams are supposed to be “self-managed”, right?  Doesn’t that imply that we stop managing? 

This line of thinking is misguided, dangerous, and will ultimately raise the probability of failure to new heights.  It might even be worse than the waterfall (command and control) philosophy of the last 30 years.  But, if we dig just a little bit, we quickly uncover in the origins of agility, through methodologies like extreme programming (XP) and Scrum, that exactly the opposite is true.  The rigor does not go away, but the philosophy changes drastically in a number of key ways.

Kent Beck, the father of extreme programming, test driven development and one of the original signatories of the agile manifesto said in his early book,

“When I first articulated XP, I had the mental image of knobs on a control board. Each knob was a practice that from experience I knew worked well. I would turn all the knobs up to 10 and see what happened. I was a little surprised to find that the whole package of practices was stable, predictable, and flexible.”

Does the phrase “Turn all the knobs on the control board to 10” sound like “winging it”?

Discipline and Rigor in an Agile Software Project

If all of this sounds like theoretical pablum, then let’s get concrete.  Below is a list of activities in a typical SDLC, followed by some practices we have found that work, and how we might “turn the knob to 10”.  To be clear about this list, some of these activities would be seen in projects of old, just not executed continuously.  Agile constitutes a mind-shift, similar to how lean manufacturing shifted thinking about building physical products.  For example, in 1970, American manufacturers couldn’t believe that building large consumer products ”to order” was reasonable.  Big batches, queued up at every step in the value stream was viewed as “efficient”.  In 2014, the philosophy has changed drastically.

(This list is not meant to be exhaustive, but simply examples to illustrate agile activities)

• Some ways Agile teams amplify Project Management
  • Continuous Planning:  Shifting the focus from a one-time uber-planning event, towards constant planning and re-planning.  This helps teams rely less on “the plan” that was developed in the beginning of the project (when we know the least), to relying on regular planning activities (as we learn).
  • Small Releases:  Planning and committing in small releases, versus big bang delivery at the end.  This step is challenging as the work to make this happen forces leaders to challenge the “big batch & queue” models from the manufacturing processes in the early to mid 1900’s.  (interestingly, manufacturing teams figured out this fallacy as early as the 1940’s and 50’s through the work of visionaries like Taichi Ohno)
  • Customer Engagement:  Ensure the customer is part of the team, or always available to the team to shrink feedback loops and allow for immediate decision making.
  • Process Tools:  Use tools like the planning game, the system metaphor, blitz planning, walking skeleton, MVP, and planning poker to ensure constant and realistic planning is short bursts
  • ROM Estimating and Error Bars:  A project managers ability to build trust by constantly delivering is important.  One risk to expectation setting is a project managers ability (and fortitude) in knowing how to be predictable using rough order of magnitude estimating and clearly articulating the delivery date “error bars” at the end of a release or project.  Even when a date is articulated and credible (meaning a date is imposed for legitimate reasons), the discipline of setting appropriate expectations by de-scoping is critical.
  • Hyper-Prioritization:  Helping teams pull critical features to the top of the priority stack continuously. Manage risk in this way so that we avoid ignoring issues until the end of a project.
  • Team Effectiveness & Psychology:  Constant evaluation of team effectiveness and health. If teams come to a standup daily and talk about missed tasks, or failure to integrate code, or failure to interface with customers, the project manager has the obligation to stop the line and help fix it.  Often, even when the question “what is impeding your progress” is asked daily, most people won’t tell you there’s a problem.  This is where the term “spidey sense” actually helps.  But it only helps when you know how listen, and evaluate reality against a set of agreed upon practices by the team.  That means that we need to expect the team to actually do this stuff religiously.
  • Vision, Roadmap & Release Plan:  Ensuring we have a vision for this project that it’s well articulated and that we have a consumable roadmap of prioritize features/stories.  This plan is constantly re-evaluated and compared against reality.
  • Risk Management:  Activities such as evaluating the interface between vendors and shared service groups within the organization. If team cannot get a shared service in the room to commit with the team and work with the team in a collocated fashion, they have just inherited a project risk and dependency activity.
  • Visual Control:  This can come in many forms, but one well known technique in IT is the Kanban Board.  Slightly different than manufacturing Kanban, the basic premise is that we need to visualize the teams work, limit their work in progress (WIP), and focus our teams on “smooth flow” through the IT system.  The biggest killer of effective delivery in a complex system are invisible queues (places where work products sits in a “wait state”).  If you visualize the steps that work items take to get through a system, you immediately recognize the issue.  You will immediately see very few value adding activities awash in a sea of waste.  Most work, when allowed to be invisible, is “sitting and waiting”.  And, the more you allow teams to multi-task, the more waste you are allowing in the system.

• Some ways Agile teams amplify Business Analysis
  • Continuous Requirement Elicitation:  Agile teams amplify requirement elicitation. We do this by forcing ourselves to understand the businesses view of the requirement using user stories, pictures, models, wire frames delivered by the real users of the request. Different than a BRD, stories allow us to understand why the customer wants with they want, and also allows us to create domain specific acceptance criteria through collaboration with them.
  • Triad Development and Collaboration:  BA staff are constantly interacting with the QA and developers and evaluating the work product as its produced.  This requires enormous amounts of collaboration and work, but the output is typically grounded in reality, versus the fiction of “the plan”.
  • JIT Requirements:  Rework is lessened as BA’s stop building huge requirement inventories that age and rot.  There is a constant questioning and “de-scoping” that happens throughout the projects as requirements are built as they are understood.  This allows the team to “maximize what we don’t build”.

• Some ways Agile teams amplify Software Development
  • Pair Programming:  This technique allows code to be written by a pair, where the “driver” writes code and the “observer” is freed to review the code being built real time, as well as help keep the strategic intent of the feature in mind while code is written.  This technique tends to build high quality, well-structured code, fast.
  • Test Driven Development:  Write a test, watch it fail, write code to make the failing test pass.  TDD helps dev teams write incredibly focused, well designed code that is also “change tolerant”.  This is incredibly important as we introduce refactoring.  Instead of eliminating change, we relish and encourage it.  We also employ the same concept using BDD, or behavior driven development where requirements are evaluated using code.
  • Refactoring:  Mentioned above.  This is critical as we want to allow business users to change their minds frequently so we build exactly the right thing.  This takes extreme discipline.
  • Continuous Build:  One dirty little secret of system development projects is that, no matter if the code builds on your machine (or in your development environment) it is NOT done.  In fact, many, many projects fail because they are never able to integrate and deploy the code.  This technique allows us to get over this hurdle early, and forces teams to confront reality very early in their project lifecycle.
  • Other Concepts:  Agile teams amplify software development by focusing on code craftsmanship, standards, pattern harvesting, and evolutionary design techniques as well as others to ensure that we are building quality into our products versus checking for quality at the end of the project.
  • Automated Testing:  Automated unit testing, pairing, TDD, ATDD, BDD, daily check-in, continuous integration, heavy interaction with BA/QA/Ops, etc are completely necessary

• Some ways Agile teams amplify QA
  • Triad Development and Collaboration:  Agile teams amplify QA by forcing QA specialists into collaborative discussions (daily) on individual user stories and features. Often you will see QA people working on acceptance criteria and helping to train others.  With a testing mindset in collaborative discussions, many features are built with the end user in mind, versus without context.
  • Automation:  QA also is responsible to set up automated testing mechanisms for the “simple stuff” so that we can have the humans move onto the complicated requirements. This requires QA to learn new techniques, become more technical, and interface often with the team (business partners included).
  • Full Team Members:  The QA staff are now full-fledged members of our teams instead of a department that gets handed a work product to evaluate at the end. This is a large shift for this group and one that forces them to be incredibly rigorous in their delivery.

• Some ways Agile teams amplify Coaching
  • Leadership & Psychology:  Agile teams amplify leadership by creating a social contract between professionally equipped teams and the goals and outcomes they are to achieve.
  • Collaboration:  The command-and-control management style tends to build teams that only know how to execute tasks and rarely improve.  This leadership model, however, is bar none the easiest to implement, and typically yields shitty results and demoralized teams. We are trying to get our teams to recognize the business need / business value and the rigor necessary to execute and to improve continually while they deliver.
  • Value Stream Management:  The value stream manager role is a servant leader position to support the team when they can’t deliver in (often) incredibly challenging environments.
  • Gemba:  Translates to “Go and See” in Japanese, this technique is critical for leadership, and one that tends to be lost in many organizations.  This concept is important as every team has different needs and contexts.  There are many developmental levels as well.  For instance if a team needs to deliver a feature using technology that is new to them, a VSM will take a more hands on approach to teaching / guiding.  If teams are well equipped for a challenge, a VSM might employ a more Socratic style to effect appropriate change and challenge the team to improve.  The only way a VSM can know what to do and how to help is to GO AND SEE.

• Some ways Agile teams amplify Operations
  • DevOps: Continuous Integration and Delivery:  Agile teams amplify the operations and deployment of software by forcing teams to integrate every single day through automated “hands off the keyboard” techniques.   Continuous deployment is delivered through a four tiered environment structure so that we can show customers real, legitimately working software every single day (Or as fast as we can deliver it)  That’s important because it allows us to shorten feedback loops to incredibly small proportions and remove waste from the system. For instance I can write the feature today, check it in, build it, have all the tests pass, deploy it, and show it to a customer and get feedback in the same day. This avoids the shit storm of integration issues most large projects suffer
  • TQM:  From an IT standpoint, the total quality movement means something very similar than the manufacturing technique of the last 30 years.  Essentially the team is committed to build quality into our systems versus checking for it at the end.  Buggy, poorly designed, rarely tested mountains of crap software are regular fixtures for Operations teams.  Using many of the agile techniques listed, the number and type of defects introduced into a production state is deeply reduced.  They are also delivered into production in tiny, easily understandable packages that are easy to remove.  The net effect is an incredibly stable environment that has reduced the risk of change to such a degree that, in many cases, the deployments can be fully automated.
  • Full Team Members:  The Ops staff are now full-fledged members of our teams instead of a department that gets handed a work product to deploy at the end. This is a large shift for this group and one that forces them to be incredibly rigorous in their delivery, as well as challenge designs of the team early and often.  This allows the team responsible for operating the system to help affect it’s design.

The following story is entitled “Three Envelopes” – the story that kicked off my foray into Enterprise Data Management:

A woman had just been hired as the new CIO of a large Financial Services corporation. The CIO who was stepping down met with her privately and presented her with three numbered envelopes. “Open these if you run up against a problem you don’t think you can solve,” he said.

Well, things went along pretty smoothly, but six months later, the organization hit issues and she was really catching a lot of heat. About at her wit’s end, she remembered the envelopes. She went to her drawer and took out the first envelope. The message read, “Blame your predecessor.”

The new CIO called a meeting and tactfully laid the blame at the feet of the previous CIO. Satisfied with her comments, the organization responded positively, things began to pick up and the problem was soon behind her.

About a year later, the company was again experiencing issues, combined with serious morale problems. Having learned from her previous experience, the CIO quickly opened the second envelope. The message read, “Kick off an Enterprise Data Program.” This she did, and the IT organization, and morale, quickly rebounded.

After several consecutive worry-free quarters, the organization once again fell on difficult times. The CIO went to her office, closed the door and opened the third envelope.

The message started, “Prepare three envelopes…”

In technology organizations, especially when technology isn’t the core business, “systems thinking” is rarely taught.  I find that interesting and deeply ironic. The more time you spend trying to help organizations transform, the more you realize how often really smart people fall victim to this gap in awareness.  The result:  much of what we call strategy is actually nothing more than ham-handed fiddling.  And that fiddling often has disastrous unintended consequences that slow the pace of change at best, and ruin careers or organizations at worst.  All this to say that in corporate IT, we need to think more like ecologists than technologists.

So Why “Ecology”?

I’ll demonstrate using a concept from ecology called a “trophic cascade”.

Question:  What is the most effective and fastest way to quadruple the size and density of Cottonwood and Aspen trees in National Parks like Yellowstone?
Answer:  Re-Introduce wolves

I know what you’re saying. “Whatchu talkin’ bout Willis?!”

To illustrate the point, below is a description of how the Yellowstone ecosystem lost Aspens and Cottonwoods (and much more) when wolves were removed:

1. Wolves were killed at scale and completely driven out of the park
2. Large herbivores, such as elk or deer, increased in number and foraging behavior changed significantly.
3. These animals over-browsed preferred plants, especially deciduous trees and shrubs like cottonwood, aspen, willow, and oaks, and spent more time in riparian areas.
4. As a consequence, “recruitment” of cottonwood and aspen (i.e., the growth of seedling/sprouts into tall saplings and trees) was drastically reduced, and uncommon plants became rare or were disappeared completely.
5. Long-term loss of streamside vegetation caused major changes in channel morphology and floodplain function.
6. Loss of berry-producing shrubs, and young aspens and cottonwoods, led to changes in the diversity and abundance – and sometimes the outright loss – of other species, including beaver, amphibians, and songbirds.
7. The disappearance of top predators triggered an explosion of smaller “mesopredators,” such as coyotes, which led to further cascading effects.

The term for this phenomenon is a “trophic cascade,” defined as the “progression of indirect effects [caused] by predators across successively lower trophic levels.”

For millennia, humans have attempted to accomplish their objectives by “fiddling” with variables using the simplistic “I want A, so I’ll change A”-style. Ham-handed as it was, I must admit, some of these issues are not simplistic at all. Who could have foreseen that killing wolves in a National Park would decimate aspen trees? Or some species of bird? Or frog?

Thinking Like an Ecologist – Indirect Effects

The item I want you to pay attention to in the above definition of a trophic cascade is “Indirect Effects“.  Ecologists understand this concept all too well.  Ecology is the study of connections and relationships, and therefore an ecological approach in business strategy forces its creators to think about the relationships a particular decision might have on the “system” around it.  Ecology is a foundation of systems thinking and helps us understand feedback loops, indirect effects, and dynamic (non-trivial) interconnected systems.

Just like the decision to artificially remove a “top-of-the-food-chain” predator has disastrous consequences for ALL species in that ecosystem, so too might seemingly rational decisions in IT strategy deeply, and negatively affect critically important teams, systems, processes, etc. (indirectly).

So What Might We Do?

Here are some suggestions to keep in mind as you build forward looking strategies in your IT organizations that uses an emergent, ecological approach to design {adapted from Emergent Design Solutions}:

• Design from top down, and from bottom up simultaneously
• Practice patience and careful observation – Only begin articulating the design when you see or experience the patterns of the system underlying the Emergent Design.
• Practice cultivating adaptive methods rather than prescriptive methods.  Discourage reliance on highly prescriptive methods that typically introduces excess rigidity into the design. Safety and rigidity are useful only as a catalytic structure for emergence.
• Ensure continued health, resilience and deepening wisdom in the system by encouraging adaptation to changing conditions within the system
• Practice Continuous Improvement – No sacred cows – Be open to questioning assumptions.
• Focus on people and their relationships rather than process – cultivate intelligence and wisdom in your human ecosystem.  Process is the guide. It is the wisdom and quality of relationships that determines the usefulness of what is produced.
• Cultivate mutualistic respectful relationships; however be aware that negative feedback loops and frictional forces are often critical to the health of a system.
• Seek to become aware of and engage wholes and nested networks of relationships rather than causal relationships and linear hierarchies.
• Encourage rapid prototyping. Fail successfully and often.
• Seek slow small simple designs over fast, large, complex ones.
• Follow the basic law of emergence – simple principles can lead to the “complex beautiful”.
• Engage diversity.

I’ve built my career being strong willed and opinionated. As I’ve grown and challenged myself (especially in leadership roles) I’ve come to realize that maintaining that opinionated attitude is the exact behavior that will keep me right where I am. Namely, stuck. Not only that, it will keep everyone else around me stuck as well.

As a younger person in an individual contributor role, it’s important to learn your trade. It’s critical to solve problems and to be “the expert”. Having an opinion, using it (and relying on it) is not only important, it’s mandatory if you want the work to get done.

There is a very different approach as your sphere of influence grows. As a leader, there is an inflection point where your staff will clearly know as much, and often more, than you do about the day to day details. It’s at that point when your opinion matters WAY less than it once did, and you need to work hard to encourage your staff to drive.

Don’t misunderstand.  There is a real need for everyone, leader or not, to share ideas, debate and explore, but as a leader, you need to be much more deliberate about when and where you share your ideas.  It’s important to create space for your opinions, be it in sessions designed for that express purpose, working with peers or with your boss, or even writing blog posts or editorial pieces on the internet.  But, in the day to day trenches, your staff needs much, much less of your opinion than you likely want to give.  They need your trust and support.

Much more than in my professional life, I have come to gradually realize the need to be less dogmatic and opinionated in my personal life as well. Maybe even more so. The people I care about need my compassion and understanding WAY more than they need my opinion.

The willingness to bite my tongue is extremely difficult, but it is the hallmark of maturity and growth to be sure. And it’s not just about “active listening”. It’s every bit as much about recognizing that importance of your interactions, being connected to someone, and that “serving” them doesn’t always require the dispensing of advice or wisdom. In fact, it likely rarely does.

I see this as a leadership opportunity in my personal life that will not only help me, but is a sign of respect and caring for the people around me.

So, I’ll leave you with words of my strongly opinionated older brother, which helps anchor my behavior whenever I feel the need to opine…

“Opinions are like assholes. Everyone’s got one…and yours stinks”

In this brilliant video, Dave Foster Wallace explains that real education is not about knowledge, but about how you look at your challenges in life and how you choose to interpret life’s annoying trivialities that are all around you. Education is about awareness.

This is his commencement speech from 2005 at Kenyon College. I’m having a tough time explaining the not-so-trivial impact this lesson has had on me.

We all have a choice. Education affords us an opportunity to reflect. We have the opportunity to choose to see things and interpret their meaning positively, compassionately. Or, we can choose to be miserable, self centered wretches, unknowingly imprisoned by what we KNOW to be true (that just isn’t). That our default condition (that WE are the center of the universe) can be overcome with hard work and deliberate effort, and the knowledge that we can choose to think differently.

Related to work and career, one thing I do know is that happiness and fulfillment is not found in your “work”, per se, but in relationships and committed, long term sacrifices for those you serve – eg people you care about. (family, friends, teams)

http://youtu.be/vET9cvlGJQw

At a recent leadership event focused on Organizational Change Management, one of the comments I heard described a key point that I felt I couldn’t let go unnoticed. The comment that was made was, “we need to continue to show empathy to our partners, teams, and peers”.

At the surface, this seems like a “garden variety” type of statement. It’s not.

At one point in the day long leadership event on OCM, I told the story of my own attempts early in my careeer related to organizational change. It was mainly about mistakes I made related to “human factors”. My failure is littered with me (as the change agent in firms) showing very little empathy. Unfortunately, I see a lot of that everywhere I go, and sometimes (sadly) I was the cause of it. It’s sad because at NO point in my past did I feel like I was doing the wrong thing. From my point of view, I was trying very, very hard to help. What I learned was how much you have to pay attention to people, their emotions, their circumstances, and definitely have empathy if you’re ever going to have a chance to help them.

Devolve Into Name Calling

I don’t like to generalize too much, but this one actually might help. This incredibly simplistic generalization is that leaders tend to self-select into 2 groups:

• Change Agents
• Managers / Stabilizers

Change Agents challenge the status quo and tend to feel good with ambiguity, breaking things apart and trying to affect the system in a positive way. Managers / Stabilizers are focused on effectively and efficiently operating the system and spend their time making everything work predictably, at reasonable cost, with low risk. Basically, change agents change broken systems and stabilizers run and perfect them. What I have learned over the years is that WE NEED BOTH. After all, you can’t constantly smash organizations to bits and rebuild everything. You also can’t “polish”systems that are deeply, or fundamentally flawed.

I have also learned that there are predictable tendencies between these two groups:

1. Change Agents tend to look at Managers / Stabilizers as incapable of change, inept, lazy, and attribute behavior to personal characteristics (e.g. those people don’t know what they are doing!)
2. Managers / Stabilizers tend to look at Change Agents as arrogant, emotionless villains who are only out for their own self-interest. Everything they do is seen as despotic and offensive. (e.g. that guy is an asshole!)

Essentially, organizations devolve into name calling and the Fundamental Attribution Error. This cognitive bias basically is placing a heavy emphasis on personality traits to explain someones behavior rather than thinking about external factors.

A Third Trait

A third trait needs to evolve if we are to ever get forward momentum in our organizations. We need Leaders. Leaders help others understand that empathy and understanding is critical, and people behave based on incentives, as well as intrinsic or extrinsic motivators. They also teach that diversity of thought is critical, and that organizations need heavy doses of both roles to be successful. Leaders help both sides (and the unlucky middle) understand why we need both types of people, and how to effectively coexist. In effect, they teach people about ecology.

I will be writing more about ecology in upcoming posts. In the meantime, let me know what you think!

As a strapping young lad, I worked for my Dad’s management consulting firm “on location”. I happened to work at a firm that made a very, very exciting product – metal drawer slides (booyah!). In its simplest form, this product has exactly 3 parts: 2 pieces of metal (bent, punched and painted) and 1 nylon wheel that gets pressed onto one end. This firm did not suffer from lack of customers or an oversaturated competitive landscape, but this firm was clearly struggling. So why were we there?

For starters, they did not really understand their entire business from end to end. They hadn’t given thought to an integrated supplier network. Partnership with the people that actually shipped their products, raw material suppliers, etc? – Uh, no.

I will write at length about the menagerie of issues they encountered in another post, but for now, let me describe ONE of the symptoms those issues caused – a massive finished goods warehouse. The warehouse housed actual finished products, buffer stocks of every product sku, and much more. It also housed huge quantities of production product runs that weren’t to spec, or were sent back from drawer manufacturers in large quantities because they “didn’t fit”. It also contained blems, discontinued product lines, and more buffers.

At first blush, I’d guess this seemed like a pretty good idea. How else would they ensure they had enough product on hand to ship to customers at a moment’s notice? Not only that, it took 2 full days to retool the machinery and powder coating bays to get ready for new product runs. So, while they had their most popular models running, they decided to build extra units to ensure they were being efficient, and didn’t have to retool the machines (and waste all that time). So, they did what many manufacturing firms did. They built a massive finished goods warehouse. When they ran out of space, they built it bigger. In my estimation, they’d run out of space 3 or 4 times before we got there.

Imagine their surprise (and mine) when the first thing I was asked to do was to take 5 friends from the shop floor and start “recycling” mass quantities of finished goods, old parts, blems, etc. It was a horrible job. It felt like I was on an episode of “Hoarders – Enterprise-style”. We gutted that warehouse over the next weeks and months, leaving one “relatively” tiny section in the back corner near shipping and receiving for storage.

Why would we do that?

Let’s start with how firms look at excess inventory and why they manage it the way they do. How firms view excess inventory often determines the practices they employ to manage it. Excess inventory, in many manufacturing firms, is viewed as:

• An Asset (because it is listed on the left side of the balance sheet)
• A Stock buffer – “insurance” for demand fluctuations
• Insurance for production problems (scrap, incorrect bills of material, shortages, on the floor engineering changes)
• Some also view it as efficient use of resources. After all, BIG BATCHES are more efficient, right?

So what’s the real impact of excess inventory? Many companies view costs as simply the cost of borrowing money to finance and build the excess inventory. The true costs are significantly higher and include the following:

• Holding costs (cost of capital, warehousing, lost items, theft, damage, insurance, physical inventory counting)
• Product Obsolescence – finished goods are at heightened risk of becoming obsolete or unneeded
• Opportunity costs (missed sales)
• Delayed shipments due to order fulfillment issues and growing complexity – sometimes requiring expeditors

When all costs are considered, most companies find that inventory costs are in the 25 percent to 35 percent per year range. Yikes. This warehouse became a dumping ground for an out-of-control process. It was simply waste, but no one understood it. They didn’t have an idea of its true cost, how to rectify it, or how to eliminate it.

So how does this relate to Software Development?

If you look closely, there are obvious parallels in the software development space. The most obvious is related to the “large batch efficiencies” firms think they get in specifying software in advance. This one is fairly obvious to most, as requirements and “intended solutions” age and become obsolete. This causes an entire raft of dysfunctions from the waste of refactoring old and outdated specs, to the worse problem of massive and onerous change control processes that tend to drag projects out for months or years (and sadly products are built that no one wants or needs). What a waste!

If we dig a little deeper, many organizations (like mine) have adopted Lean principles and are attempting to build “flow” and “pull” into our processes and teams. We construct Kanbans to visualize that flow. By in large, this simple visual approach allows us to “see” bottlenecks at every steps in our process. This is HUGE because we can now take action to remove the bottleneck and get prodcuts to flow.

Interestingly, it ALSO allows us to see where we build “buffer stocks” and “finished goods warehouses” in the value stream. These buffers are often NOT acted on as we rationalize away the dysfunctions as “necessary” to our big batch mentality. This is why it’s important to challenge the need and inspect why you have them.

My opinion: Sometimes software-centric “finished goods warehouses” ARE necessary, but mostly they are built to cover up an out of control process that no one understands, all under the guise of “big batch” efficiency.

What we are trying to teach now, and what teams need to start to challenge themselves to understand, is that a “buffer stock” or “finished goods warehouse” can create the very same dysfunctions in your development processes as bottlenecks. It hinders flow, hides issues, and can create massive waste.

So….you are seeing the bottlenecks. Can you find where you build “buffers” or “warehouses” and explain why you need them? Do your kanban boards display excess inventory anywhere – even if you made an active decision to put it there? Is it more important to have them in the process than building flow? Are you building those things into the process to avoid another issue?

So What’s The “Big Idea”

Many of us, including me, work like the dickens to create systems that filter and classify work in our professional lives. Filters that allow us to get things done (execute), and think strategically. It’s critical in business to sharpen our ability to be planful and take advantage of strategic opportunities, and equally important to execute. If we don’t, everyone who counts on us will suffer, including our organizations broadly. The good news is that we have decent feedback loops in a business setting, and we get that feedback fairly quickly.

In our personal lives, this attitude of “plan and execute” tends to change. Somehow we feel that we go home to “get away from all that”. We don’t want to take our work home with us, so we divorce ourselves from the lessons and techniques we use at work when we arrive at home. People think (mistakenly in my opinion) that nothing should cross the work/home boundaries, lest our lives get thrown irreparably out of balance.

It’s no wonder that most people put little to no effort into building a reasonable sense of agency – the ability to think strategically and act on those strategies – when it comes to their own personal lives. My wife and I have come to realize that this is a huge mistake, because the things that we want to accomplish in the pursuit of fulfillment in our own lives tend to come from a category of stuff that we call “life goals”.

So to begin, I want to illustrate how “life goals” differ from other activities.

Define Your “Stuff”

As we take on the challenges of our days, work (or “stuff”) comes to us from everywhere. This “stuff” falls into the Stephen Covey Urgency/Importance Matrix, and can be defined this way:

Distractions - (Low Urgency, Low Importance)
These are items that are not important, nor are they urgent. They tend to be things that pop up all the time, because we let them. For me, these are things like web surfing, short stints in front of the television, arranging papers on my desk, browsing a magazine or two. It’s simple trivia and busywork.

Interruptions - (High Urgency, Low Importance)
These are urgent, but hold little/no relative value. I have three kids, and my life seems to be filled with these. “Dad. Dad! DAD!!!!” my son will scream from his room. When I arrive in a panic, thinking he just broke his leg, he asks me to get him some batteries for his flashlight (or a “peanut granola bar”).

Life Goals – (Low Urgency, High Importance)
This is stuff that’s important, but there is no urgency to get it done. This is where our “life goals” come from. This includes stuff like proactively building strong family friendships, reading and reflecting on parenting techniques to help our children build their independence and life skills, and creating a vision for what our marriage could be as we get to know one another more deeply, and deliberately creating the life we want.

Critical Activities – (High Urgency, High Importance)
These are things that are really important, and have clear urgency. These situations can take the form of emergencies, urgent family matters, disasters, etc. The typical response is to summon any and all resources you can to handle the activity, dropping everything else in your life. You essentially enter autopilot mode and go.

To build a life of fulfillment and meaning, my opinion is that you have to spend as much time as you can in quadrant #3 – Life Goals. Of course we need to spend time in all four of these quadrants, but spending time in the other three is not hard. In fact, those quadrants tend to “just happen”. Busy work, interruptions, and emergencies come and go all the time. And if you let yourself, you will stay in those three areas (and grow very little). Only by building your agentive “muscles” can you think about those areas that will actually create the life you want.

OK. By now you’re saying that talk is cheap. So, to put my money where my mouth is, I decided to take a few key lessons from my professional life and use them in pursuit of happier, more fulfilling days with my wife, my family, and myself. Interesting things ensued! I am learning a great deal about the power of taking my own advice. I’d like to describe the process along with some of the lessons learned (more details in a subsequent post).

The Concepts Used

Specifically, we used a planning retreat, a concept for perpetual planning that works extremely well in work settings. The concept is simple enough. Take yourself and your significant other “offsite”. Go somewhere familiar or go somewhere novel, but find a place that you can get some privacy, some focus, and get to work. Organize your time and prepare for the day with homework. Schedule a follow-up session quarterly. If you’re really aggressive, schedule a weekly check-in.

There are more than a few powerful concepts that emerge from time like this. Some examples include concepts taken from Lean and Kanban. One example: the ability to talk about emotionally charged areas of our lives with partners with a little less emotion. After all, if we have thoughtful life goals (about things like budgeting, marital communication, our kids, etc) and they are part of our quarterly offsite discussions, then we have a non-emotional way to bring these issues up in relative safety, and we can do it on a cadence (versus when we bounce a check) **I feel compelled to tell you we don’t bounce checks. :)

Other concepts examples: treating the “act of planning” as much more important than “the plan”.

“Plans are useless, but planning is invaluable” – Dwight Eisenhower

Via Personal Kanban, we are attempting to get everything out of our heads and visualized to manage the “existential overhead” of keeping tasks and projects in our brains. We combined GTD concepts to put the outcome of our planning in a trusted system so we could “manage the system, versus manage the work”. We determine things that need to get handled NOW, versus things we can PLAN, or those that we can DEFER (or things we can THROW AWAY). We use dates and deadlines sparingly for goals. We have plenty that has to have a date associated with it, but the bulk of our goals have no date because we view those dates as needless constraints and a source of guilt and angst if we find ourselves “living life” and missing occasional deadlines. Plus, we don’t want to be “shackled” to dates if we feel we can deliver faster (ok, I threw that last one in as a challenge to myself)

Lastly, we are using agile principles as well and have created a regular cadence to reflect on our approach (and ourselves) and tune accordingly. One way is through a weekly chat that allows us to review our system. The second is a “re-planning” session that will happen quarterly. If we start to come off the rails, or we feel like we are moving in a direction we don’t like, or we find ourselves with a new challenge we didn’t expect, the mechanism is in place to help us take deliberate action.

Well, that’s it for now. You’ve caught the experiment “in progress”. I would like to describe the actual planning process in more detail and I will do that in the next post. Maybe we will have a result by then!

Recently, Jim Benson asked if I could participate in an upcoming interview series on his blog called the “Modus 3-Question Series”. I unwittingly agreed. :)

Below are my answers to those interview questions.

Q1: Do deadlines empower your teams?

Unfortunately, the short answer is YES, I do feel deadlines empower our teams, but NOT in the way you might expect. Deadline empowerment can lead to wildly varying, and often times negative, outcomes.

To begin, most organizations don’t have a good sense of how to manage software developers or “value” in software projects. So, instead of working on a way to lead more effectively, they use a very handy (and simple) proxy for leadership…deadlines.

For most business leaders who don’t have the time, patience, or inclination to learn about software development, understanding the nature of these projects, and all the variables at the disposal of project teams, is daunting. In many organizations, much of the complexity is ignored, or worse, dismissed. In these organizations, software teams and their importance in the business ecosystem is marginalized and quickly these teams become more order-taker and less partner over time.

So then, what do deadlines “empower”?

Development teams, like any team of people, are motivated intrinsically to do a good job. If “doing a good job” becomes more about WHEN you deliver over WHAT, HOW, or WHY you are delivering, then those teams will move Heaven and earth to deliver ON TIME, ignoring some key concerns.

Common missing components in deadline-driven projects:

• (HOW) The deep complexity and craftsmanship needed to build a sustainable, resilient, manageable product that will grow and thrive over time.
• (WHAT) What is being asked of us? Should we build it all? Some of it? None of it?
• (WHY) Why are building this? Is there a better way? Can we help make this any simpler?

When teams are faced with a deadline, the typical response is to focus on the deadline and deliver (no matter what). This is definitely empowering, and seems like a reasonable arrangement, until you realize the incredible cost associated with missing the WHAT, HOW and most importantly the WHY.

(I can explain those costs more fully in another post.)

Q2: Do your customers expect delivery on deadlines?

Yes they do. And sometimes it is completely rational and unavoidable. Market pressure, response to regulatory mandates, legal requirements, etc., all require rapid, time bound responses to delivery.

However, in my experience over the last 17 years in (primarily) IT shops in financial services, insurance, banking, and even packaged software products, these deadlines are largely imposed by managers as a proxy for leadership. Customers and IT managers simply don’t know what else to do.

Q3: What are you doing to meet customer expectations (in response to or in lieu of deadlines)?

We are attempting to teach a new way of working. There are a few key steps we’ve taken to help our organization realize that deadlines, although sometimes very helpful, can also cause mayhem and colossal long term costs if placed in the center of our work. These costs can be predominantly avoidable.

Here are some key tenets we try and aspire to:

• Ensure your decision makers (the “business”) are engaged and partnering daily, and understand tradeoffs related to their choices. With good decision making and partnership, we tend to maximize what we don’t build, and eliminate waste.
• Educate the organization on the virtues of software craftsmanship, and why high quality software is crucial to reduce costs and to go fast. Architecture is then “baked in” to the process and we find ourselves leveraging our software versus fighting it.
• Teach the organization to “shrink their bets”. Massive projects take on a life of their own. And living things tend to create antibodies to change. With large projects, too much is riding on the vision, plans, and promises of the project and its stakeholders. When that happens, human nature is to “press on” and achieve project deadlines (even if the project or company is destroyed in the process).
• Teach organizations that learning is the most crucial, and scarce aspect of projects, not money or capacity. With smaller deliverables, teams can learn how to deliver high quality software extremely fast, and learn a great deal along the way. Rapid learning allows teams to change course quickly and achieve stunning results in the process, without massive investment in expanded capacity. (we do more with much, much less)

When the above conditions are met, customer expectations move away from simplistic models where DATES are the only thing they understand, to active partnership where RESULTS and VALUE once again take center stage. It’s not that deadlines are all bad. We understand they will always have a place in our work. We simply try and make them less important in our daily lives, and treat them as exceptions rather than the rule.

When we do this effectively, we find that deadlines take their rightful place in the background of our work versus in the center of it. The transition is tough and takes time. Over the years, we have found it becoming easier and easier to implement due to a thoughtful reflection on the deadline based alternative – years of deadline driven management and all the dysfunction and unintended consequences it has created.

I love the word “failure”. I love it! Why? Because it lets me know that teams are trying to get better! People are human, and everyone will fail (with some regularity in complex situations – I might add). To me, an awareness of failure, and open dialogue about failure is crucial for teams to begin to take ownership of committments, and their own evolution. But there’s a problem.

In my experience with some agile frameworks, failure is a nasty little concept. It doesn’t have to be, but sadly it tends to turn out that way.

The story goes like this:

Someone (me in a few regretful cases) enters and organization and creates a buzz around new concepts like Scrum and XP. We form teams and have lots of earlier struggles and, conversely, lots of early wins. During the process, the concept of time boxed iterations (sprints) is taught and internalized. Sprint planning commences and sprint goals are agreed to.

Thoughtful leaders keep these goals from becoming a list of user stories to be delivered. They design goals to be (struggling for a word here)… “thematic” (meaning, the goal is something a team and a business can sink their teeth into and is the living embodiment of the features, stories, and tasks that make it up the Sprint).

Fast forward to the retrospective:

The retrospective kicks off and Sprint Goals are discussed. Inevitably someone says, “we sort of delivered the goal, but we aren’t DONE. We missed 2 user stories. We committed to 10, and we only got 8 of them done. How can that be a success!?”

This is typically followed by a giant sucking sound that, over time, turns out to be the souls of the people that worked tirelessly to get 8 pitiful stories completed. (the sounds of them leaving their owners)

The game is now afoot:

What is a typical Pavlovian response to this kind of interaction? If I rap your knuckles and called you a failure because you can’t plan in 2 week increments, you have 2 reasonable “protective” responses.

1. you learn how to actually predict the future, and only commit to work you KNOW you will deliver, or
2. you sandbag in an attempt to ensure success and your own self preservation.

In my experience, working with brilliant, thoughtful, honest (albeit human) teams, the typical response is #2…. They commit to 6 stories, “success” is restored, and the balance in the force is regained.

What just happened?

My guess is that most teams, as they mature, could begin to deliver huge volumes of value…more than they ever thought possible in the beginning. Unfortunately, when teams link story count and timelines to failure, those outer boundaries are unachievable. Sadly, we have just created a team that will rarely, if ever, commit to more than 6 stories. Mostly it’s because of an incredibly myopic view and definition of “failure”.

Teams internalize failure as “not delivering the requisite number of stories on time”, no matter how hard you try and convince them otherwise. In fact, I have NEVER met a Scrum team that defines failure as anything other than not delivering enough on a predetermined timeline. That’s it. That’s all I hear. Ever…. Ever-ever. Really.

That’s not to say that teams don’t fail for many other reasons, but those really AWESOME failures (the ones we need to discuss and learn from) are rarely talked about, and (as I said two sentences ago) are never internalized as the actual “failure”.

To me, the interesting things that happen are when we try new techniques and they don’t work out, we implement strategies that need a little tweaking, we build a product that no one will buy, etc, etc. Those are awesome and can teach us a lot, and need attention. Time-boxed approaches make discussing these types of failures difficult because most are focused solely on the stop watch.

In Conclusion:

The fact that you cannot tell the future as a team is not revolutionary, interesting OR relevant, so let’s adopt an approach that doesn’t tie “professional guessing” (even in small time boxes) to our shared definition of success. For good measure, I would love to hear some of your examples of sprint failure, or your opinions on the topic! For extra credit, discuss how lean thinking and flow-based models help alleviate some of the problems caused by time-boxed approaches.