Why We Abandoned Crowdsourcing

Todd Sullivan

We created Flightfox so experts could help you travel farther, wider and cheaper. At the time, we banked everything on crowdsourcing because it seemed the most effective way to find the best flights. Now, 18 months later, we no longer believe this and so we've launched an exciting new product. The results are so much better; here's why.

By Todd Sullivan · February, 2014

Sign up your company now!

Guaranteed Savings · Instant Setup · 24/7 Concierge

Alternative Formats

At Flightfox, we've tested the following formats:

  1. Competitive Format - many experts working against each other
  2. Collaborative Format - many experts working with each other
  3. Consultative Format - one expert working with one customer

The competitive format gets the most attention because it seems the most fun, but after running 17,000 crowdsourced contests, we saw more potential in the other formats.

The Case For Competition

We initially chose the competitive format for the following reasons:

  1. It aligns everyone to a single goal (e.g. find the lowest price)
  2. It maximizes effort by offering a single prize

These points were certainly true to an extent. For example, one expert built a round-the-world itinerary to six continents all in business class for $2,500. Few people would take his hectic itinerary, but he got the best price and hence won the prize.

The Case Against Competition

That previous example highlights our most critical discovery:

In a contest, the expert works to win the prize, not satisfy the customer.

On paper these should be equal, but in practice they often aren't. We found the following unfortunate consequences:

  1. Experts often only help while they have a good chance of winning
  2. Experts often work against the customer to win the prize

This means customers are on their own. They must work hard to keep all experts working towards their interests, not just the prize. This is no easy feat... but there's more.

The Nature of Flight Search

Searching for a flight is similar to searching for a fugitive. The expert plans their approach then spends most of their time searching deeper. They may learn something surprising and adjust their strategy, but each discovery typically makes the next action more accurate.

A contest doesn't work like this.

In a contest, experts don't work together. They do a lot of the same work and make the same mistakes. The thrill of the chase may get them working a little harder, but that's not enough to compete with formats where experts learn from every previous action.

Improving Flightfox

We knew we could do better, so we focussed on these objectives:

  1. Generate the best savings
  2. Provide the best service

While a contest could generate good savings, it was the worst format for providing the best service. Remember, the customer has no advocate in a contest. This doesn't just manifest as fewer pleasantries, but a brick wall blocking the best results.

So collaboration seemed the obvious choice.

But we questioned whether experts would share their secrets and work well together. Plus, who would get paid? If the fee wasn't shared equally, wasn't it just a contest all over again?

In testing collaboration, we learned the following:

  • Experts actually want to work together, but
  • The user experience crumbles as more people get involved

It turns out competitive crowdsourcing creates a contrived form of animosity (between experts and customers too). In the right environment, experts would rather be friendly and work together.

This was great news, but collaboration proved too awkward. "Too many cooks in the kitchen", came to mind. Experts would ask similar questions, customers would give conflicting answers, and so we experienced the same issues all over again.

A Reluctant Test

We were reluctant to test the consultative format because consulting is uninspiring. While people use the term crowdsourcing with wonderment, they use consulting with derision. Again, we persisted and found the following:

  • Experts attended to customers in seconds, not hours, and kept working in harmony
  • Customers received personalized service and their satisfaction shot through the roof
  • Experts now acted like they owned the customer and even initiated their own refunds

Our Net Promoter Score (NPS) went from 20% straight to 60%.

Everything negative about other formats seemed solved, and everything we presumed negative about this format didn't eventuate. Could this be true?

It seems we unwittingly benefitted from the following:

  • By giving the customer only one expert, we greatly improved the experience
  • By running the trial with only our top experts, we maintained high quality
  • By ditching the contest format the process now made sense

Another very strange thing happened: experts kept collaborating.

With very few experts in the trial, they didn't need to compete. They chatted throughout the day in a private chatroom and discussed how to give customers the very best results. Unlike before when everything was a secret, they were now learning at a rapid pace.

The Launch

We took a leap of faith, spent the holidays writing new code, and then boom! We launched.

Everything is better. If you use Flightfox today, you are guaranteed one of the world's best flight experts. And you will receive their undivided attention until you leave happy.

Behind the scenes, they'll consult with other experts too. You should see how courteous they all are in our expert chatroom, it resembles a romantic comedy.

But we know you just want the best trip with the lowest prices, and that's what this format provides. The contest format was fun, but the new format is clearly better. We love it, the experts love it, and according to your ratings, you love it too.

If you haven't already...

comments powered by Disqus
We just refuelled the jet. Please click here to refresh and continue.