As I understand during A/B testing users are randomly shown slightly different layouts/visuals/UI/something and after some time the company decides which one performed best and uses that in the future. Is this correct?
Then how do they determine which is better without asking the users? How does Youtube know that I hate it when they don’t show me when the video was uploaded when I watch videos anyway, since that’s why I went there in the first place?
In: 2
They might ask users, sometimes you visit a website and get those survey pop ups? Those surveys are on way for companies to gather user feedback.
Otherwise they can gather data on how users engage with the new site. For example how long a user spends on the site, which buttons they click, etc. They then use those metrics to decide what choices to make. Keep in mind what they are aiming for is not necessarily what you are. Unfortunate but true.
In the case of YouTube, they don’t care if you like the visual change. They care if the visual change causes you to watch more videos, so they can monitor that metric without asking you.
A/B testing is done using software (VWO, optimizely, Google tools) that tracks which users got which variation and tracks various goals/conversions…
Let’s say you have a landing page with two variations, trying to get potential customers to sign up for more information. The page has a video about your product, some information, and a lead form.
You might look at how many people clicked the video, how long visitors stayed on the page, how many started to fill out the form, how many submitted the form. Form submission is the most important stat, so you’d look most at that one as primary goal.
But you might then even take it off-line and see whether the page w/ the higher form submission rate ultimately converted into higher customer acquisition — maybe the page with the higher submission was because people just wanted some freebie or contest entry but weren’t qualified leads, while the other form captured fewer leads but they were likely potential customers.
Latest Answers