Three important lessons in checking assumptions and testing quickly
This essay is about our recent shift to rapid testing to validate ideas away from assumption-based evaluation, and how that helped our organisation re-evaluate some core offerings and saved us a ton of time. We will go over the role of connecting the dots between serendipitous events, how we tracked the value of what we implemented, and the importance of documentation, all of which were small events that turned a small initiative into a catalyst for a much broader change.
Recently at District 3, we’ve been doing a lot of rapid product testing through the Particles Design Lab. One of the things that came out of it is actually this Medium publication itself which has been a success so far based on the metrics.
Another service we’ve launched recently is called the Creative Dose, and the original purpose of it was to be another 6–12 week program for our entrepreneurs at District 3. We currently have three programs that are running which address areas that entrepreneurs need help in achieving: “Business Model Validation (BMV)”, “Minimum Viable Product (MVP)”, and “Product/Market Fit (PMF)”.
The Creative Dose, initially called the “Ideation Program”, was to come before BMV to help people get ideas for businesses that they can then go and validate. The issue that inspired this was that aspiring entrepreneurs were being admitted into our BMV program with ideas that weren’t developed enough to actually even begin validating.
Together with Rachel, the two of us set out to start planning an entire curriculum for six weeks of intense brainstorming, design thinking tools, and a variety of creative exercises. The intention of this curriculum was to ensure that people had a clearer idea before they joined our BMV program, and we started working on it immediately after being told that was what we needed.
What we didn’t do was question that assumption. We fell into the trap of adding another motor to the boat without checking to see if the holes in the hull were actually plugged. Our focus was simply on the fact that the entrepreneurs’ ideas were insufficiently formed but not why it took us at District 3 more than five weeks to realize that despite the fact that they are presenting to us their progress every Monday.
We fell into the trap of adding another motor to the boat without checking to see if the holes in the hull were actually plugged.
Connecting the dots between serendipitous events
Fortunately whilst all this was happening, at the Design Lab we had already done a few impromptu one hour meetings with some entrepreneurs who needed help with what they supposed was a layout problem, and left the session realizing that they needed to revise the entire purpose of the product. We hadn’t thought much of those conversations at the time, because it was simply the way we question other people’s assumptions as part of our trade. The people that we hadn’t been talking to were the people in the BMV program, and we discovered the reason for that was due to an overload of work that we give them to deliver weekly. This meant that they were able to stay in the program, but left them without enough time to seek out other services at District 3, and more importantly without enough time to critically reflect on their own idea.
In a sprint retrospective meeting at the Design Lab, we were discussing all the things that went well in the past two weeks, and the stories of those impromptu meetings came up as a good thing we had done. Call it magic, or simply our brains finally seeing a pattern, but it became crystal clear to us that we needed to formalize those one hour meetings and publicize them more clearly to our community. We also realized that the one hour session was essentially a one hour version of the ideation program, so why not use it to validate the need for an “Ideation program” at all? We all got excited, we called it a “Design Dose” (yeah our naming conventions change fast) and rolled up our sleeves to get to work!
We pulled out a light CSS template, created a website with a pretty URL, implemented a free off-the-shelf booking system, and then circulated this internally with a small value proposition of what people should expect when they come to us. On top of this, we told the program coordinators to specifically send us the people they believed should be in the future ideation program. Within a day, we had our first two sessions that week booked!
Tracking the value of a newly implemented idea
So far, everything sounds like it’s going great but we haven’t even met our first “client” yet, so we were nervous – it didn’t matter that the service was free or that these people were already part of District 3, we had important questions to answer. How are we going to prove that this service is any good? How can we ensure that we’re validating whether or not an ideation program was needed?
Part of the process of running this “Design Dose” involved a follow-up email that contained all the things we had discussed, along with links to relevant resources. The email was concluded with a thank you, and a link to a feedback form that asked them to tell us what they thought of the session, and we ensured that email was being opened by making it highly valuable to them – alleviating them of the need to take notes in the session and to instead focus on their thinking since they will be getting a follow-up.
We ran the first two sessions and sent out the follow-ups, and immediately realized the true value of what we were proposing. We had simply given people a platform to air out their thoughts, and helped them connect the dots between these thoughts to reveal what they would have known had they been given the right tools and amount of time to reflect.
To validate this further, we retroactively sent emails to the people in our first few impromptu talks before the Dose was created, summarized what we had discussed and asked for feedback and sure enough, the same rang true. We were helping people validate some core assumptions they had, and making them avoid days of wasted work in the wrong direction.
A quick revision to the name away from the word ‘Design’, because of the graphic connotation it had, and thus the “Creative Dose” was born. We sent out a new announcement to the community with a slightly tweaked value proposition, and we had not only a whole week and a half of sessions booked, but we also ran a few sessions outside of our working hours because the notion of reflecting on one’s thoughts was in desperate need.
I’m happy to say in just the past three weeks since the Dose launch, we’ve hosted over 12 teams (we have four weekly appointment slots for the Doses – so yes, we are literally overbooked), and gotten back over 8 responses to the feedback form, all of which were positive, many of which contained helpful tips for us. Best of all, it meant that what we’re doing has concrete evidence of its value and, in the greater scheme of things, this meant that we plugged a hole that was missing in what we had already been offering.
The Creative Dose, which was meant to be a bandaid solution to an issue in an existing program had graduated into a service that was actually valuable and worth pursuing, but what did it mean for the “Ideation program” that it was initially meant to be?
The discovery that the missing component was that we needed to do more listening on our part, and provide time for reflection on the part of the entrepreneurs, effectively invalidated the need for yet another program. This has saved us hundreds of hours across our team of coordinating yet another program despite our already low resources.
R.I.P Ideation program…
To many people this may be giving you an anxiety attack. What about all the work that went into the Ideation program? How do you just start a service and ensure it stays good?!
Answer: A million and one things, we didn’t say this was easy – but we guarantee you documentation will go a long way!
Documentation as the core of repeating a success
An unintended benefit that came out of co-writing the follow-up emails to the Creative Doses on Google Docs before sending them was that we had unwittingly created a log of what we had conducted during each hour.
This is now proving invaluable in repeating the Doses and training other people to provide a Dose outside of just Rachel and I. It also became a base of information that would later inform a variety of new developments at District 3.
One of the things we are working on based off of this information, for example, is developing a structure for the Creative Dose to make it even more helpful and reduce the time it takes so that we can service more people. Another is in the development and revision of the core tools at District 3 that we offer our entrepreneurs, since the Dose emails reveal patterns of what is being asked of us and we’re learning which tools are actually effective thanks to the feedback.
Note that documentation doesn’t have to mean data hierarchy, a massive wiki, and some insane info-system procedure. We’re simply writing about what we’re doing, taking photos with our phones where it makes sense, and sending out surveys. At our small level, this is more than enough. We can take care of truly institutional levels of documentation as we grow, but the overhead of a big system would have hindered our performance as much as not documenting anything at all would have prevented us from repeating our successes effectively and reflecting on the failures.
The emails, photos, and surveys being documented are also making us far more transparent by allowing us to show our team the activities we’re conducting and the results of those activities. It’s also a great way to transfer knowledge across teams in our organisation as well as publicly to the world!
Even the documenting of our energetic curriculum for the original Ideation program proved to be beneficial despite the cancelation of the program. It guided the curriculum for a unique internship that we’re relaunching this summer called the Innovation Projects. The Ideation program lived on in spirit thanks to a file saved on a server somewhere!
A big system would have hindered our performance as much as not documenting anything at all would have prevented us from repeating our successes effectively and reflecting on the failures.
This is not to say that everything is going perfectly. In the past three weeks, we’ve experimented with providing these sessions outside of our standard four slots, conducting them in a variety of rooms, structuring them in a variety of ways, and even bringing in people from outside of District 3 to get a completely different point of view. Each of those variations had within it certain elements of failure and success that has either been deprecated or replicated with each subsequent dose.
The insights from our work and documentation with the Dose has revealed the importance of introducing some critical offerings to our incubation. The entire District 3 team is currently involved in a fairly large project that involves rethinking what it means to incubate an entrepreneur and what exactly are the tools needed for true innovation to happen.
Am I saying that this is all thanks to a tiny, one-hour consulting initiative? Of course not.
No idea can catalyse change in a vacuum. We simply took a rather small leap of faith and executed on a spark that came out of a few good realizations and conversations, rather than prematurely killing the idea because we’re afraid of failure and wasted time. We would have wasted hundreds of hours more had we just blindly created a program that proved itself not needed in just a few hours of implementing a little service, which would have ultimately resulted in a massive failure.
No idea can catalyse change in a vacuum.
The only real risk we took with testing our idea the way we did was a few hours of our time, and 5kb of space on our web server. At worst, if it proved unpopular we would kill the page that took at most an hour to build, and only two people would have ever even heard of the service before it disappeared into the ether of our community’s memory (though it would never disappear from our internal documents – tracking failure is as much part of documentation as tracking success).
When assumptions are being made about problems, the ideas generated to tackle them, no matter how small, should be tried before they are killed. If the right idea collides with the knowledge of a talented team at the right time, its implications can go from servicing two people to becoming its own product, and inspiring change across the organisation.