AB Testing for YouTube Thumbnails

128 viewsOther

I’m not asking what AB testing is, I am asking how the actual statistics work. I’m going to use MrBeast as my example.

MrBeast switches up his thumbnails sometimes up to 5 times a video, most people being under the impression that it’s for AB testing. I’m aware that AB testing is meant to figure out which thumbnail gets clicked on the most. What I don’t understand is how this data is even verifiable or accurate in any way considering the amount of variables:

* The 10mil people (example number) who saw the first thumbnail are completely different from the set of 10mil who would see the second thumbnail.
* Those first 10mil people who already watched the video are very likely not going to watch the video again after the changed thumbnail.\*
* The YouTube algorithm more than likely has a large hand in who gets recommended certain content.

My main confusion is from the first two points. How are you going to determine what thumbnail works better if it’s not the same set of people choosing between two thumbnails? In my mind, the way I see a definitive answer of a choice between two things is to have one group of people choose between those things. But if you have two different groups (or more), and the first group already watched the video so they’re not going to click the second, I can’t wrap my head around how this is even useful information.

In terms of YouTube videos specifically, I imagine most of the views come pretty much within maybe 12 hours after posting, at least with MrBeast. After that, I imagine the growth of views starts to stagnate at a certain point or even slow down. This is another major reason why I don’t understand how AB testing works, because the data from the 4th thumbnail change is probably in their “slow down” phase of video views, so again, not understanding how it’s accurate or useful info.

\*To further my second point, let’s say MrBeast’s team expects that he will get 500mil views on his video. With the first thumbnail, 100mil people watched it. That means 100mil people are now removed from that pool of potential people clicking on the second thumbnail, because it is under the assumption they don’t plan to rewatch the video. So the statistics are now dwindled to 400mil who can click on a thumbnail vs 500mil.

In: Other

3 Answers

Anonymous 0 Comments

>My main confusion is from the first two points. How are you going to determine what thumbnail works better if it’s not the same set of people choosing between two thumbnails?

All thumbnails are served at the same time, using YouTube’s built in “Test and Compare” tool, just to different audiences. One randomly selected audience will see one thumbnail, another randomly selected audience will see one thumbnail, and so forth. What it does is compare how many people saw a specific thumbnail on a listing and end up clicking through to watch the video. The audiences are obviously unique for each thumbnail, but that averages out with volume.

You are viewing 1 out of 3 answers, click here to view all answers.