A Keystone Habit in Engineering
Shipping is one habit that unlocks many others
In the book, The Power of Habit, Charles Duhigg describes the concept of a Keystone Habit as the kind of habit that creates a ripple effect in our lives by triggering other habits in its wake. A positive keystone habit is therefore a multiplier that helps us pick up multiple other useful habits along the way.
The habit of shipping is one such keystone habit when it comes to building software. Developing the urge as a team, and working in unison, toward shipping your changes to production (and therefore to your customers) as often as possible happens to be a powerful habit. As you aim to develop this muscle, a number of other habits need to fall in place.
Enough talk about habits. Let’s talk fundamentals.
How can you ship your changes faster?
Velocity is often not the ideal metric of progress. Let’s think about turnaround time for a change to get deployed to production instead.
Smaller Changes
Logic dictates that smaller changes should take a shorter time to prepare, validate and deploy. In order to ship more often, breaking your work down is the obvious first step.
Making your changes smaller however is easier said. What appears to plague most teams is that they often get into the habit of breaking up all work into projects.
This is not a problem in itself. When you measure progress by completed projects, the team tends to build each project as a whole across weeks or months before finally testing and deploying in one go. I like to call this the Big Bang Deployment strategy.
Research proves that larger and longer running projects are more prone to failure. If your intuition tells you otherwise, question that with data.
Just how do you make your changes smaller?
Let us inspect this question at both the macro and micro levels.
Macro: Requirements and Scope
Product requirements are notorious among engineers for what we like to call scope creep. Protecting scope while you work on the requirements is necessary but not sufficient. It is equally important to negotiate scope at the outset with stakeholders like Product Managers and clients so that one can agree on a minimum viable version of scope (often called an MVP).
The goal should be to get the MVP out there, with instrumentation for measuring success metrics, and start learning from users what works and what doesn’t. As an added benefit, by reducing the turnaround time for the change, you drastically reduced the opportunity for scope to creep in before the MVP goes live.
Now that you negotiated an MVP, it should be a default to work on iterations of the product or feature once the MVP is out.
Micro: Iterations
Iterations do not begin after the feature is live. Breaking up an MVP into deployable iterations that beautifully combine by the time the final deployment goes live is essentially software art all great teams must learn. A feature flag is the easiest trick in the book to deploy changes silently into production, until one fine day you turn the flag on and the magic is unleashed.
Note that each deployment needs to be testable because if you plan to test only at the final juncture, splitting up wasn’t really worth it.
Automation
So we have learned to meticulously plan and break down our work into small changes regularly being prepared to go to production. Preparing a small change is however only half the challenge. Getting it deployed with confidence is when the value is realized. Confidence comes from the fact that few, if any, bugs are found and the change is deployed flawlessly, without side-effects, to production.
As humans, we excel at creativity, but when it comes to doing repeated, mundane tasks we tend to drift and falter. This is why the drudgery of integration testing, regression testing, and build and deployment, is best left to automation. Done well, not only is it faster, but also more reliable and predictable.
The quest for shipping more often will naturally push teams toward greater reliance on automation.
Automation is not a destination
Automation is a journey, and all great teams must undertake it. While their paths may vary, the journey usually involves suffering the travails of manual software testing and deployment, and addressing the most painful problems one after the other by leveraging automation.
The early pains are usually associated with human error or mind-numbing boredom where you begin to ask yourself - “Why are we doing this?”. The outcomes of early automation are usually relief and confidence, not speed. Over time, the flywheel of automation begins to turn through feedback, fed by past victories, before automation begins to breed more automation.
Welcome Side Effects
Armed with small changes and automation, your team begins to ship regularly. Look closer, and you will start to see a number of other improvements all around.
Quality
The smaller surface area of a small change reduces the likelihood of bugs, while automation additionally helps eliminate common, repeated errors. This naturally leads to software of higher quality.
Once again, don’t believe your intuition. Just look at the research.
Experimentation and Innovation
Armed with the habit of scoping tightly and measuring your success using metrics, teams are more prone to experimenting and taking calculated risks. If a change does not work out, given its small size, rolling back or simply accepting the sunk cost is not a big deal.
Momentum
Each deployment, following the last one in rapid succession, becomes the heartbeat of your team. The feeling of forward movement is palpable and things start to add up very quickly as you iterate. The snowball begins to grow in size and starts to become unstoppable.
Predictability
As engineers, we are generally over optimistic when it comes to predicting when we will be done. The larger the change required, the worse our estimates are bound to be.
While large projects have been either massive write-offs or completed with over 100% overrun in time and resources, our tiny steps toward MVP, and the steps that follow, end up being a lot more predictable. Confidence in engineering predictability improves and business is able to make plans around when a product or feature would land.
In summary, by simply focusing on developing the habit of shipping regularly, a team can learn to -
Have raging debates on what an MVP should constitute
Chop changes down to the smallest possible testable and deployable chunk
Iterate rapidly past an early version reacting to incoming feedback
Appreciate and rely on automation over human endeavor for repeatable tasks
And as a result, begin to -
Demonstrate a visible increase in software quality
Experiment and innovate more often and more boldly
Enjoy their work through the momentum they see all around them
Be a lot more predictable with timelines for delivery
If that isn’t enticing enough to make shipping your primary goal, what is?