VWO is an Experience Optimization Platform that improves key business metrics by empowering you to easily discover insights, test ideas, and improve engagement across the entire customer journey. In this fascinating interview, CEO Sparsh Gupta reveals some priceless secrets to getting A/B testing right and shares his personal insights on what he sees as the evolution of user experience optimization.
Please describe the story behind VWO. What started the idea, and how has it evolved so far?
VWO started as Visual Website Optimizer in 2009, back when A/B testing was a fairly new concept, and there was hardly any software in the market. The concept of optimization existed, but people who searched for A/B testing would only find scholar articles or academic papers that were lacking a practical solution that marketing teams could adapt. We saw that some of the very large companies (e.g. Google, Amazon) were doing a lot of A/B testing, but the technique was completely missing in the workflow of the masses. The results gained by those doing A/B testing made us realize its potential, and we decided to take the concept to the larger market.
We were also clear that the reason A/B testing was still alien in the SMB/MidMarket segment was that the concept was very theoretical, and it wasn’t easy to conduct an A/B test. We decided to build a product that allowed people without any expertise in statistics, or knowledge of how websites work, to use the technology. The task ahead was to build an easy-to-use product, and that’s exactly what we targeted.
We launched one of the easiest website editors (which was visual and hence became a big hit, fortunately), and looking back, I believe that was one of the key reasons why the industry transformed in the last 10 years. The editor made it very easy for anyone (non-engineer) to make changes on a website in an easy manner (almost like editing a powerpoint presentation).
Since the concept of A/B testing was new, getting engineering resources was difficult for people who wanted to give it a try. We wanted to remove all friction points, and therefore, aimed at making the testing process so easy that anyone could do it end-to-end without involving their IT/development teams.
The market really liked our initial innovations, and the product was a hit, but the customer set was very small and new, so we had to find ways to expand the base in order to grow further.
There was growing competition in the advertisement space (including digital ads), and the bids for limited ad slots inventory was constantly increasing. It was high time that companies started focusing on increasing business from their existing traffic and not just on growing the traffic (by spending more on advertisements). A/B testing solves exactly this problem, and all we had to do was educate the market.
Our only GTM back then was producing a lot of content in order to educate our potential users and inform them about the concept and benefits. We spent a lot of time educating and growing the market. As more and more people started reading about the concept, they also signed up to use our product. This is how we grew almost 100% year on year for the first few years. There was strong customer advocacy, and internally, we were focusing on making the product more comprehensive and easy to use for the masses.
Once our product was feature-rich (we launched multivariate testing, split URL testing, segmentation, personalization, asynchronous tracking code, etc., many of which were the first in the industry) and the market was growing (thanks to all the awareness around), that’s when we started analyzing how our end customers were using the product. We noticed that while some of them were doing great, many were just not getting it right. We started spending time thinking about how these companies could reach results, and we found that for a successful A/B test, what one needs is a well-researched hypothesis (i.e. idea on what to test). Random ideas tested do not give the expected results and often disappoint/demotivate the user.
The right way to come up with a good hypothesis is to analyze end customers’ behavior and user analytics data. This data has insights on how the end-user is interacting with your websites and, if planned well, it can give a lot of insights and highlight gaps which can be tested for more wins.
If you replicate a generic test, your chance of succeeding is very low. Moreover, if 5-10 A/B tests yield no tangible results, the motivation of the whole team goes down, and they abandon the idea of A/B testing. Therefore, coming up with behavioral data-driven hypotheses is necessary for successful testing. This led us to move from being just an A/B testing software to providing a complete conversion optimization platform.
We created a conversion optimization platform that combined the power of our testing capability with behavior analytics capabilities such as heatmaps, session replays, form analysis, scroll maps, and a suite of other products that we released along the way.
We built funnels to understand user behavior, track what people are doing, where they are clicking, where they get stuck, what makes them bounce off, and what converts them. We created those products so that our customers could really understand user behavior nuances and come up with better hypotheses and observations that can then be tested and optimized.
This led to an increase in the success rate of tests for our customers and a lot of successful case studies. We collated all of them and launched IdeaFox – a portal where you could search for A/B testing ideas. The premise for launching IdeaFox was that one could get inspiration for testing, but the final hypotheses had to come from one’s own data, and so they needed to take time to understand it in order to get value from their campaign.
IdeaFox was warmly accepted by the industry, and we saw a lot of other players coming up with similar products that aimed to quicken and ease the process of A/B testing.
To uncover real insights, data has to be collected at different stages of the customer journey. The trick is to find products that help you understand what’s going on. There are many insights tools out there, but often, the data collated from different tools for analysis doesn’t make much sense because each tool has a different data definition.
The VWO Experience Optimization Platform maintains a single source of truth for visitors, and that data holds constant no matter which VWO product or capability is being used. This allows growth and optimization teams to spend less time reconciling data between products and more time confidently deriving insights on customer behavior. With this, VWO lays the foundation for brands to deliver great digital experiences at every touchpoint across their entire customer journey. We expect that this will enable brands all over the world to significantly improve their existing optimization practice and achieve faster growth.
About 50% of our customers were successfully following a set CRO process which we created for them, and getting far better results and ROI than people who were not doing it.
While we tried to push it further, conversion optimization in 2017 was all about optimizing the properties that you own, such as your website or mobile app. However, what happens to the user when he is not on your digital property also matters. If you look at conversion optimization, a lot of it revolves around user experience, and user experience goes beyond your website. That’s when we started working on a new product we call VWO Engage, which focuses on how to optimize UX when the user is not on your web property and keep them engaged with automated marketing campaigns through web push notifications & Facebook Messenger.
So let’s say you go to an eCommerce store to buy a new phone. A solid CRO strategy could help deliver a seamless user experience, but in many cases, you might still end up abandoning the cart. Follow-up emails make a good way of getting users back on the website, but the experience has to be continued in further communication because these are very isolated channels.
We also realized the industry was facing those challenges of how you bind experiences of different users across their journey even when they are not on your website or app anymore.
That, to a large extent, is what we call experience optimization. I believe that the entire industry is moving in this direction. It started from A/B testing and has now shifted to optimizing end to end experiences, user interactions, and specific messaging per each scenario, which is what personalization is all about. In the last 10 years, that’s how we have been doing it, so you can say VWO is now an Experience Optimization Platform.
Here’s a sneak peek into the VWO dashboard:
What are the most fundamental things that every digital marketer should know about Customer Experience Optimization?
One thing is that there has to be a strategic process and timeline around experience optimization. I’ve seen thousands of customers and marketers trying the ad-hoc approach. Once in a while, they will run a few tests for a campaign, or think about optimizing their user experience, but will fail to get results and abandon it altogether.
The entire experience optimization process should be seen as a continuous program. The thing about A/B testing is that if you do it today, you need to do it again next month. The world is moving so fast that even if your website experience is top-notch right now, customer behavior will evolve drastically in the near future and you need to be on top of things virtually all the time.
What I’m also seeing, from an A/B testing and CRO standpoint, is that getting more winners is tougher. When I say winners, I mean campaigns where the hypothesis was verified and validated. There are also campaigns where the hypotheses are false, so moving from 50% to 60% is hard.
You need to accept the fact that more than half of your campaigns will not result in winners, and that’s okay; it happens to some of the most optimized websites. Even the most sophisticated campaigns fail by more than 50%, so you should not give up. The idea is to constantly keep testing.
I’ve seen teams that focus on a number of tests and experiences being able to optimize better over time and get more results than those focused on getting just one test right. Even if you do a lot of planning, there’s still a probability of the test failing.
You need to go with a data-backed hypothesis, but don’t be extremely detailed about the hypothesis because it’s the testing velocity that matters. Having more campaigns is far more important than just sticking to one and perfecting it.
As far as experience optimization is concerned, it’s not just about A/B testing to create superior customer experience on your website; it’s the entire communication with that customer across all touchpoints that matters. It can be your website or app, social media, emails, or any other touchpoint that impacts the customer experience and affects your conversion.
How would you advise marketing professionals to deal with the huge amount of big data being streamlined from analytics tracking tools?
There’s undoubtedly a lot of data that information analytics products collect and show, but a marketer needs to understand what information is relevant.
You need to look at data in two ways.
One is exploratory, where your aim is to learn/identify potential problems in your system (customer experience in our case). Usually, you don’t know what you’re looking for and are just trying to figure out if there is something you are not aware of. There are certain data sets that really help with that aspect. You need to look at data from the perspective of learning and understanding the users better. If you spend time looking at data from session replays, heatmaps, funnel drop reports, form analysis, chances are you might get a hypothesis like “Oh, I see a lot of people are not clicking this main CTA”, “People are not even scrolling to the section which has my main discount coupon”, etc. Now you have a problem to solve.
The second approach is when you have a problem identified and are exploring potential solutions to that problem. You can rely on the same data and a lot more analytics to understand the underlying reason that caused this problem.
Certain data will show you the problems, but will not tell you why it’s happening. Other bits of data can help you understand why things are happening.
If you understand what the analytics reports tell you, you can make a lot more sense of the data. Having said that, it’s also the responsibility of companies like ours to simplify all this information and data into something meaningful & actionable. We’ve been attempting to do that for the past ten years.
In your view, which technologies can we expect to see more of in this field in the next 5 years?
I’m a strong advocate of data science, AI and machine learning which a lot of people have been talking about. These will take center stage in the next few years. In our field specifically, these technologies are still young, but I believe that in the next few years, there will be a lot more use cases around these emerging technologies.
Let’s say you own an eCommerce website, selling sunglasses in France. With so much data out there, someone should be able to tell you how to design your website, what image to put on your homepage, how your pages should look, and what content you should include. Currently, it’s the marketers who need to do all the processing: understanding the data, coming up with hypotheses, implementing them, interpreting the results, and giving it to the IT department to take forward. With data science and AI, this process will be automated, with some focus on auto personalization. Marketers and engineers will just need to throw different variations to the system, and it will automatically personalize and optimize itself continuously for every individual visitor.
Personalization is used on a segment of users. Let’s say that people coming from France on a Sunday make one segment. But, within this segment, there are a lot of different people that should see a different personalized version of the website. For a website that has 2M views per month, it would be impossible to comprehend these micro-segments and personalize the experience for each. This is where machine learning and AI can help create a reality where each customer coming into the website will be treated differently; displaying an experience that is right for that individual. I believe that’s where the world is moving, and it will do so in an automated manner.
I also believe that in terms of marketing, a lot of data collaboration will happen. Data is being collected by different tools and products but they do not seamlessly integrate with each other. It’s a big pain point for marketers because while different tools give different perspectives on data if all this data was integrated into one uniform platform, it would have a lot more value. I believe the entire industry will focus on this integration in the coming years, creating a more holistic representation of the end customers.