The 11 Best A/B Testing Tools in 2020

Besides being really fun, A/B testing is a way everyone involved with business – whether you’re a product manager, marketer, analyst, or user experience designer – can both create impactful innovation and mitigate risk while doing so.

A/B testing (i.e. running a controlled experiment, normally in a digital environment like a website or an application) is prevalent nowadays. All of the biggest tech companies you know – Microsoft, Netflix, Booking.com, Google – run thousands of experiments per year. Now it’s weird if you *don’t* run A/B tests.

While many of the most sophisticated enterprises have teams of dozens or more working on home-built testing tools, even the smaller shops can get started relatively easily nowadays.

There are dozens of tools on the market dedicated to A/B testing, in addition to hundreds of other marketing solutions that have built-in A/B testing as an integrated feature. This means you can run experiments on landing pages, Facebook ads, product features, sales scripts, customer service prompts, and literally any other thing you can think of in a business context.

You do, however, need the infrastructure to do so, meaning you’re almost always going to go through the shopping and review process to find a testing tool.

While there is no “best” A/B testing tool, there are better or worse solutions depending on your specific context. This article hopes to match you with the right tool for your needs.

It’s mainly based on my experience, but I’ve also had a million conversations over the last several years and am bringing in some opinions and consensus, as well as objective feature and price comparisons when I can.

The 11 Best A/B Testing Tools in 2019

  1. Conductrics
  2. Optimizely
  3. Convert
  4. VWO
  5. SiteSpect
  6. Kameleoon
  7. AB Tasty
  8. PlanOut
  9. Wasabi
  10. Google Optimize
  11. Adobe Target

1. Conductrics

Conductrics is my favorite experimentation platform. I like the CEO, Matt Gershoff, and he’s been an excellent industry mentor and friend to me. He is also one of the most thoughtful leaders in the analytics and optimization space, so you can trust the direction of the platform.

He has also built a really great piece of software that, in many ways, was far ahead of its time and definitely ahead of most of the others on the list.

Power/feature-wise, it’s the best you can get. You can run A/B tests (client or server side), multivariate experiments, and bandit tests, all from a WYSIWYG editor (if you want). You don’t need to install a script until you set things live, as you can use the browser extension to edit and preview variants.

Additionally, the tool lets you run predictive targeting to exploit different segments of your audience. Basically their AI will discover which combinations of visitor traits respond best to each of your content variants, automatically.

Definitely a tool for true optimizers.

2. Optimizely

Optimizely is the biggest brand name on the list, and I have to say, I’ve always loved using the platform. It’s what I learned on, so I’m definitely most comfortable in this interface, even to this day.

Feature-wise, they’ve really improved in the last few years, especially in their enterprise offering. They’ve now got ‘full-stack’ experimentation capabilities, valuable for those who want to run server-side experiments or run tests on IoT, TV, or mobile apps. The also offer unlimited free feature flags and rollouts.

Overall, it’s a robust and powerful enterprise solution that even features a fully fleshed out program management platform (though unfortunately the company doesn’t really maintain support for smaller companies nowadays – it’s really only at tool for large brands).

Their stats engine is based on sequential testing, which is often more efficient, but I do love a bit more flexibility in terms of statistical analysis.

Optimizely is probably in that tier of software where, like IBM, ‘you won’t get fired for buying Optimizely.’

3. Convert

If you want almost all of the features of Optimizely (with the addition of truly wonderful customer support and a free CXL Institute membership), but at a fraction of the price, Convert is the way to go.

For starters, the interface will feel familiar and intuitive to experienced experimenters. For newbies, too, it’s not a cumbersome tool to learn.

Convert has all the typical functionality you’d expect from a client-side A/B testing tool: A/B tests, split page tests, multivariate, and personalization.

What I like most about Convert is the founder and the team clearly truly care about the industry and their customers. They’re much more transparent about their features and the market, and I find the lack of marketing spin quite refreshing (see: their feature comparisons).

Plus they offer a free 15 day trial, so you’ve got nothing to lose from checking it out and seeing if you like it. I’d recommend those currently only looking at Optimizely or Adobe to take a look and see if Convert.com might meet your needs as well.

4. VWO

VWO is similar to Convert in that it’s much more affordable than Optimizely (though VWO has gotten more expensive in recent years), and it has many of the fundamental features you’d want. Of course, you can run A/B tests, split path tests, multivariate tests, and personalization targeting.

You can also use their other CRO features, things like surveys, session replays, funnels (and the rest of HotJar’s feature list, basically). If you like all-in-one, VWO is the way to go (they even offer solutions for server-side and mobile testing).

They offer other less-CRO-related tools, as well, like push notifications.

Overall, their CRO-suite is top-notch and feature rich, and I love using VWO. They offer a free 30 day trial, so again, you’ve got nothing to lose from checking it out. VWO is an excellent option for optimizers at all sizes of companies.

5. SiteSpect

SiteSpect has been around the space for a while, is well-respected, and is generally thought of as the go-to server-side A/B testing tool. They do, however, offer many other testing solutions, including your typical client-side website testing.

Additionally, they offer great ecommerce optimization features like behavioral targeting and recommendation engines.

Performance-wise, this seems to be one of the fastest testing tools on the market. It’s also got a reputation to this day of being one of the most powerful and robust platforms, giving in less often to randomization errors, flicker effect, and other factors that could affect external validity.

6. Kameleoon

Kameleoon is a great experimentation platform hailing out of Paris, France but serving customers everywhere. They are known for their powerful personalization capabilities, but you can get all the basics done as well (AB, MVT, etc.).

As part of their personalization offerings, they have predictive targeting (let the machine find proper segments to exploit), business logic targeting, email personalization, content personalization, and segment analysis. They also have a feature called KCS (Kameleoon Conversion Score) that weighs the relative probability that someone will convert (likely a productized version of propensity modeling).

It’s one of the more robust personalization packages, and top it off with a trustworthy A/B test delivery system, and you’ve got a great tool.

Seeing as personalization is one of the easier things for marketing leaders to talk about and imagine, and one of the harder things to actually implement in a meaningful way, this platform could help you walk the talk.

7. AB Tasty

AB Tasty is another France-based experimentation platform, this one very powerful and feature rich.

For marketing teams, they’ve got your basic client-side A/B testing and multivariate experiments, as well as personalization and social proof notifications for online retailers.

For product teams, you’ve got robust server-side experiment delivery, feature flags, and progressive rollout. They recently launched a whole new “suite” for product teams called Flagship, marking their seriousness in providing full stack experience optimization capabilities and supporting product teams and engineers.

Great full-stack experimentation and optimization platform.

8. PlanOut

PlanOut is Facebook’s open source testing framework. It’s a fairly simple framework that helps you randomize (through deterministic hashing) and automatically log important events.

I’m a fan of using open source solutions or even building out your own basic platform, especially if you have advanced or unique use cases or at the very least own all of your raw data and have greater control over the deployment and analysis of tests.

9. Wasabi

Wasabi is another open-source experimentation tool, this one by Intuit. It’s simple to set up and 100% API driven (instrument in whatever language you’d like).

This one also comes with a nice, flexible UI too get you started quickly, in addition to real time metrics and reporting.

Also, look at it like this: it was built for and is used by one of the more sophisticated experimentation teams in tech, Intuit. Clearly it’s powerful and flexible enough for their myriad of products and use cases, so I’d imagine you could make it work for your org.

10. Google Optimize

Google Optimize (the free version) is a free, yet very limited, experimentation tool. It lets you run simple client-side A/B tests, as well as multivariate tests and split page tests, though you’re capped at 5 tests per container. You can seem to get past that by spinning up multiple containers (you could in the past at least, haven’t used in a while).

For some reason, Google Optimize uses both opaque observational units and an opaque statistics engine (some flavor of Bayesian, I’ve gathered). It’s some weird amalgamation of users and sessions, and totally unclear to me still.

I like the tool as a cost-free solution to randomize users in experiments, but if you’re using it, I’d log events to an external data tool and do your analysis there.

The biggest benefit to Google Optimize is the native integration with other Google tools (namely Adwords and Analytics). This allows you to push variants to custom Google Analytics segments and remarket them with display ads (just one idea). The integration is really powerful.

I can’t speak to the paid version, but the free version is great for getting started but probably shouldn’t be used past a certain level of program sophistication.

11. Adobe Target

Adobe Target is the enterprise powerhouse of the experimentation world. Even today, they still power most of the largest brands and websites, and they still have some of the most robust features of any testing tool.

As with Google Optimize, Adobe Target’s strongest benefit comes when you’re using the rest of Adobe’s Marketing Cloud, particularly Adobe Analytics. Again, you can push segments to be targeted via specific variants and also analyze your tests more deeply in Adobe Analytics.

This creates powerful potential for personalization.

Candidly, I don’t really have experience using the platform, though I know a ton of people who use it. I’ve also heard great things about the platform (especially its targeting and segmentation capabilities) from smart people I respect like Andrew Anderson. However, I don’t have direct experience on it, so I’ve gotta throw that out there.

In fairness, if you’re considering Adobe Target, a listicle probably wasn’t going to sway you away from or towards the tool anyway.

Conclusion

There’s no shortage of A/B testing tools on the market, and there’s something for everyone.

If I’m working on a large website and have an abundance of resources to work with, I’m choosing a tool like Conductics (or perhaps, given the organization, Optimizely, Adobe Target, or Google Optimize 360).

If I’m looking for a more affordable, but still powerful solution, Convert, VWO, SiteSpect, Kameleoon, and AB Tasty are all absolutely fantastic tools. They all have their nuances, but I can recommend any of them.

If I’m planning on building out my own custom testing solution, I’m probably at least taking a quick look at PlanOut and Wasabi first.

Tons of A/B testing tools, no excuse not to run tests!

Alex Birkett
Alex Birkett is a Growth Marketer and Content Strategist based in Austin, Texas. He's a proud UW-Madison graduate and enjoys craft beer, lifting weights, and sailing.

One Response to “The 11 Best A/B Testing Tools in 2020

  • John Bacon
    5 days ago

    Don’t forget about Sixpack by SeatGeek!

    100% free and open-source, server-side or client-side depending on your choice, and works with any language.

    I’m still frustrated by how I haven’t found a better replacement in my role. Pay tens of thousands of dollars for a poorer and less flexible client-side experience in Optimizely instead? Nuh uh, that deserves to get you fired.

Leave a Reply Text

Your email address will not be published. Required fields are marked *