What I Learned From an Unsuccessful Launch: Vym

As of today, I am gradually decreasing my work on Vym. I would like to share my journey of building it, and some of the lessons I learned the hard way.

If you would like to know what Vym is, I wrote another article about the project and my moviation behind it, you can read it here. In short, Vym is a code review tool that adds slide shows to GitHub Pull Requests.

Why Quit?

I am no longer working on Vym because of two reasons. First is that I was not able to get a single active user since launch. Either the idea or the execution was flawed. Second reason is related: the project does not motivate me any more.

I will write about these reasons in more details. But let’s have a look at how I got here.

Story So Far

In Februrary, I began working on Vym as a side project to solve my own problem that I encountered so many times during code review process at work.

On April 2, I shipped the initial version of the product.

Basically, it was a real time slide show engine for GitHub Pull Requests. I actually made a demo video which I used to present Vym at a local technology meetup in Sydney.

Shortly after I ran some user testing and found out I needed to make things simpler. So I dumped most of features and built a dramatically simplified version that works instead as a Chrome extension.

Drawings I made to conceptualize Vym as a Chrome extension

In case you are wondering how it works, here is a demo video. I actually used this pre-recorded demo to pitch Vym at a startup event:

And some weeks later, here I am. So, what have I learned?

Lessons

My learnings from this project are mainly non-technical.

Quantify and Optimize

First of all, I learned that I need to quantify my users’ actions as much as possible. That way, I can try to improve user acquisition in a more scientific manner.

Doing so is especially important when launching a new product because it is likely that there are so many holes in the conversion funnel. If you have data, you can turn them into actionable materials rather than shooting in the dark trying to get more users.

A drawing showing the conversion funnel for Vym

After launching the Chrome extension, I was not sure why I did not have any users. Jack from Trioxis, who had just started mentoring me, drew a funnel above to explain to me that I need to measure each steps leading to conversion and fix the broken part.

I actually did not have analytics software properly set up when I was wondering why I did not have any users. No woner I felt lost. There was no actionable metrics. That day, I configured analytics as soon as possible and set myself a tangible conversion goal.

Ship Quickly and Avoid Perfectionism

I tried my best to avoid over-engineering my application from the beginning and just ship a working version. It was very hard for me to fight the urges to make everything perfect (naturally because what if Vym turns out to be the next Google?)

My effort to avoid perfectionism was worthwhile because I ended up throwing away most of the code I wrote for the first version of Vym. Since my app is open source, you can actually see how much I threw away by comparing the old repo with the new repo.

Every moment of building Vym was a constant trade-off between optimization and shipping speed. It was sort of like walking on a rope.

For example, things like this often went through my mind: “Okay, if I abstract this bit into a module and rewrite these parts, I can make this code much more reusable and modular. But my users won’t care about what my code looks like and I need to ship. Let’s pass.”

Making an intelligent decision on this trade-off is hard. You don’t want to end up with a total mess of an unmaintainable spaghetti code after you ship. Yet you don’t want to take forever to ship a product. Making such decisions is a real skill, and I think it can be best learned by shipping a bunch of projects.

There is another fine line that we need to be aware of. It is a fine line between a crappy app and a working MVP. Pieter Levels puts it well in his book makebook.io:

Making a minimum viable product means it has to be viable, it’s not an excuse to be lazy and make something half-assed. So it’s all about balance. Make something great that functions and it can be minimal. But make sure it’s great.

The boundary between ‘half-assed’ app and a minimum viable product is not well defined. An MVP is made of decisions, decisions of what to throw away and what to get right. I think it takes a real skill to get those decisions right, and again, such skill can be developed by shipping lots of products.

Interview Users

After shipping the initial version of Vym, I was not quite sure what to build next. I stumbled upon a book called Sprint by Jake Knapp and used some of its lessons to design user interview sessions.

Basically I made a google form with some questions to help me filter the applicants that best matched my user persona. The form consisted of 3 simple questions. I contacted the applicants that had all the right answers I was looking for, and ran the test.

User test application form

I used a simple screen recording software and my laptop’s microphone to record the sessions. I asked the users to think aloud as they used the app, and constantly interrupted them with open-ended questions such as: “What goes through your mind when you see this part?” “What do you expect to happen when you click this button?”

First time around, I offered $50 iTunes gift card for an hour of my users’ time. When I planned 30-minute tests for the second version of Vym for $20, many people at my co-working space came up to me and offered to help for free. Shout out to Fishburners community.

I think it is desirable to have testers who are not expecting to get paid. Behavioral economics shows that when a market norm interferes with social norm, relationships become largely transactional and human elements suffer. Through the support of the community, I was able to get more genuine feedback. Oh, and for free.

All in all, interviewing users helped me identify the need to pivot, and ship the second version. Had I not learned from the potential users and kept on building what I wanted to build, I would still be stuck with the old version.

Spend As Much Time Building Traction as Building Product Itself

I did not want to build and launch something without any traction. The reason is that after the excitement of launch, trying to shift gear from building product to building audience kind of drains my energy. From my experience, doing so feels as though I am starting from the scratch after all the work I put in.

From the get-go, I tried to allocate as much time building traction as building the actual product. Thinking back, I find that I spent around 30% of the time building traction.

In the beginning, I set up the official blog for vym blog.vym.io and started publishing the story of building Vym.

It turns out I did not reach a whole lot of people via this channel. I often had to push myself to write those articles because almost no one was reading them. Yet, in the long run, those articles would be a much more genuine way to engage with the potential/actual customers than some automated tweets.

Among many methods I tried to build more traction, one that proved very successful was open sourcing the application.

I noticed that many people were looking for tutorials and code example for the application architecture I am using (It is called Mantra). After some consideration, I decided that open sourcing the app makes sense for the business.

Traffic source for Vym showing the effect of open sourcing the app

I open sourced Vym and posted links at places that harbored people who might be interested. I have gotten most of the visitors and signups from those websites, namely crater.io, reddit.com, and forums.meteor.com.

My users and I had a win-win relationship. I received many thanks from the community, and got many visitors as well. And the users finally had a practical example to learn from. If we bring something to the table instead of blindly asking for what we want, we can get create value more easily, and for everyone.

I think that open sourcing, just like blogging, is sort of a long term traction building activity. People might not start paying for your product right away, but they might remember the product in the future and think positively of it because of the value they once received from it.

Another thing I tried was making a embeddable badge for GitHub README for repos using Vym.

I embedded them in some of the open source repos that I am maintaining. It turns out this was not very successful. It may be that no one really reads README. I know I don’t.

Also, I pitched Vym at Meteor Sydney and Fishburners. It was fun, but I did not get any signups from pitching. Again, pitching turns out to be a long term traction building activity (By the way, my slides for those pitches are available here).

Provide Value Right Away

It is very important that users have to be able to use the product themselves and see what it does as quickly and painlessly as possible. A landing page with demo and sales pitches is not enough. The users need to experience firsthand, without much friction, what your product can do.

Drawings I made to try to optimize user flow

After some user testing, I realized that visitors have to jump through so many hoops before they could actually see a slide show button in their GitHub Pull Requests.

Of course, my landing page has screen shots of what the app looks like. But I noticed at one point that none of the users who signed up proceeded to create at least one slide show using Vym. Demo was not enough. Users needed the actual experience. And clearly there was a big flaw in the onboarding process barring users from experiencing the app.

My main user persona is a developer who is struggling to review a large GitHub Pull Request. Once she lands on Vym’s landing page, she needs to be able to see a slide show on her Pull Request as quickly as possible, or she is likely to drop out no matter what the landing page shows.

Note that my potential users might already be frustrated because they need to review a big Pull Request. Last thing they need is spending ten minutes setting up a new tool and figuring out how it works.

With this reasoning in mind, I eliminated the whole set up process and allowed users to see a demo version right after they install the extension.

Concluding Thoughts

We all should build things that motivate us. After unsuccessfully launching Vym, I no longer feel motivated to continue with the project.

A good thing is that my main focus has been to ship fast and test on actual people. I was never secretly perfecting code architecture or test coverage. Therefore, I do not have much emotional attachment to the project and feel free to move on.

My thought process above illustrates an underestimated benefit of shipping quickly and avoiding perfectionism. Doing so creates less emotional baggage you carry with the product. You can objectively see what is working and what is not, and decide without much hesitation to kill projects that are not working out.

But is abandoning a project based on a lack of motivation a right move? Some might argue that building a business based on one’s motivation is a rather romantic, naive concept. My attitude might be mocked as just another “Follow your dream” cliche that is so prevalent today.

Yet, I firmly believe in a mantra of a similar ilk, and it is in the form of “Follow your motivation.” We are living in a time when doing what gives us meaning and what market demands from us are ever so aligned. Building things that motivate me gives me meaning. And there is a market for those things.

In this light, I would be a fool not to follow my motivation, whether intrinsic or extrinsic.