In 2018, we ran over 654,680 experiments, with trained external Search Raters and live tests, resulting in more than 3234 improvements to Search.There are so many experiments going on all the time, the actual numbers are sort of mind blowing. Here are the stats for 2018:
- 595,429 search quality tests (the number of tests they have designed to run in the search engines. Not all of these made it to the next stage.
- 44,155 side by side experiments. The search quality raters team checks the results of old and new algorithms side by side and assesses the quality of the results, which then goes on to choose the right algorithm to deliver the best results.
- 15,096 live traffic experiments (where Google actual releases the algorithm change to the public search results and looks at how the wider audience views them, mainly through A/B testing. Some of these are rolled back at this stage. This is why we see dramatic fluctuation on a daily or even hourly basis!
- 3,234 new algorithm launches! These are the changes which are finally rolled out.
Last Updated on