Innovation at Dimagi, Part 2: How We Increased the Value of Failure

---

In this series, Dimagi’s Co-Founder & CEO Jonathan Jackson (@jonathanleej) and Senior Director of New Business Shabnam Aggarwal (@shabnamaggarwal) discuss how we continue to look for ways to innovate with the help of a dedicated entrepreneur within the company.

In this piece, Jonathan explains how we reformed our approach to innovation and exploration at Dimagi and how it gives us greater confidence in the results of our tests.

---

“‘Fire Bullets, Then Cannonballs’ is a concept developed in the book Great by Choice. First, you fire bullets (low-cost, low-risk, low-distraction experiments) to figure out what will work—calibrating your line of sight by taking small shots. Then, once you have empirical validation, you fire a cannonball (concentrating resources into a big bet) on the calibrated line of sight. Calibrated cannonballs correlate with outsized results; uncalibrated cannonballs correlate with disaster. The ability to turn small proven ideas (bullets) into huge hits (cannonballs) counts more than the sheer amount of pure innovation.”  

After reading Great by Choice in 2014, this concept framing resonated, and we started referring to our idea testing internally at Dimagi as “bullet tests.” We have always had a lot of innovation and exploration happening at Dimagi, and this framing was appealing to better organize our limited resources. 

Dimagi had not historically been great about identifying when to stop pursuing certain ideas. Warren Buffett highlights this as a key differentiator in highly successful people: “The difference between successful people and really successful people,” he says, “is that really successful people say no to almost everything.” We originally thought the bullet test framing would help us better align on whether a particular idea is good or bad for Dimagi to pursue. However, the ideas themselves were nearly impossible to prove or disprove as subjectively good or bad. Is expanding into education and agriculture a good idea? It all depends on the execution and opportunity cost.

In hindsight, many of the “bullet tests” we thought we were running weren’t “low-cost, low-risk, low-distraction” tests at all. This was extremely distracting work I was asking numerous people to do, hoping it would work. 

In 2017, Dimagi was structured by function (tech, sales, ops, professional services) that matrixed together to execute our products and services. So our flavor-of-the-month innovation ideas didn’t just require one team, they usually distracted several teams. Cross-team collaboration was also very challenging to coordinate since the idea would be everyone’s side-project priority and some were more passionate about it, while others were just trying to help out their teammates. The odds that two teams had time and sufficient interest at the same moment was hard, the odds of four teams at once was nearly impossible.

When something didn’t work, we would often claim the test was inconclusive because we didn’t spend enough time or didn’t execute correctly, leaving me with a nagging desire to try it again in a different way. This was frustrating and inefficient for everyone. Many of our areas of innovation weren’t gaining traction, but we also weren’t gaining confidence or evidence to shut the work down and move on. We were low-value failures. 

When Shabnam and I started our new business team three years ago (as she wrote about here), there was a key missing factor in our innovation effort that I didn’t realize would have such a huge benefit: Increasing the value of failure. 

By having a dedicated team at Dimagi that had all the skills needed to test new ideas around sales, tech, marketing, and services, we were creating the capability to have high-value failures when we did fail.  We would be more confident in the result. It wouldn’t fail due to lack of access to a certain skill set, or lack of focus, because it would be Shabnam and New Business’s top priority. 

One of the first ideas we had was around making a big change to our CommCare partner program. Shabnam did dedicated research and built out a business model. She did significant research on other SaaS companies’ models, their approaches, and what they wrote about publicly on ingredients for success. She compared this to our recommended change and created strong evidence to shut down the idea before even testing it in the market. 

After she presented to our COO, Carter Powers, and myself, we were all confident we should kill the approach and not focus on any changes at all to the partner program at that time. Carter was thrilled to realize, “Oh this is going to be great, this is how we get Jon to get better at saying no and shutting down ideas.”  

With our new business team, we not only created a new cannon altogether, but we also created a much more efficient and powerful calibration test when firing bullets. We brought all the skills needed into one team in order to not just succeed better, but to fail better.

---

In the next post in this series, Shabnam discusses the role of new business initiatives and how to weigh their objectives against the mission of the organization overall.

Read more here >