New Search Engine Market Share Data from Cloudflare and Wikipedia

• Accurate search engine market share data is essential to evaluate regulatory efforts to tackle the Google Search monopoly and encourage competition in the industry.
• Common sources of search engine market share data suffer from significant methodological deficiencies and vary widely for smaller search engines, creating confusion.
• Two new sources of search engine market share data from Cloudflare and Wikipedia use more accurate methodology.
• We recommend Cloudflare market share data for regulatory use, as it draws on direct referral data from millions of sites.

Governments, researchers, and policy makers need accurate market share data to evaluate search engine market diversity (or lack thereof). As explained by our series of posts on search engine choice screens (also known as preference menus), a well-designed choice screen could significantly increase competition and give users meaningful choice and control. However, without accurate search market share data, it is difficult to assess whether a particular choice screen is effective overall or to ensure consumers are presented with the search engines they want to use.

Common sources of search market share data, like the often-cited comScore and Statcounter, vary significantly for non-Google search engines which creates confusion around search engine market share. Additionally, both these and other commonly cited sources have significant methodological deficiencies. In short, comScore suffers from panel selection bias, e.g., privacy-conscious users are unlikely to agree to be surveilled by comScore and Statcounter’s core flaw is that it uses trackers, which are often blocked by tracker-blocking tools, either by search engine apps and extensions (like ours) or by other common apps and browser extensions. And both comScore and Statcounter reports are further flawed because they either do not report and/or do not have a sufficiently large and representative sample of users across all major markets and platforms.

Recently, two new market share reports were released by Cloudflare and Wikipedia respectively. Unlike comScore, Cloudflare’s and Wikipedia’s reports do not suffer from panel selection bias since they are not based on panels but instead based on traffic referred to Cloudflare-hosted websites and Wikipedia, respectively. And unlike Statcounter, this method also means Cloudflare’s and Wikipedia’s data is not affected by tracker-blocking tools. While Wikipedia is just one site, Cloudflare’s report is based on a large swath of the global Internet (25% of the top million websites use Cloudflare) so sample size isn’t a problem.

For these reasons, we recommend Cloudflare's report as currently the best source for baseline assessments of search engine market share and for assessing the effect of competition interventions like search preference menus. Wikipedia’s report is also useful because it can be analyzed in unique ways (more on both reports below). However, despite the methodological differences between all these reports, all still show that Google dominates the search engine market.

Cloudflare’s search market share report

Cloudflare's report is based on referrer data from search engine link clicks. When you click on a link from a search engine and visit that website, the site will know which search engine domain the user came from (using referrer information, e.g., duckduckgo.com). This report is made possible through Cloudflare Radar, a free public tool that lets anyone view global traffic as well as security trends and insights across the Internet as they happen. Cloudflare Radar is powered by the aggregated traffic flowing through the Cloudflare network. Radar insights like these are created by looking at patterns derived from aggregated data that has been anonymized, and so does not contain any search queries or personal information. (To be clear, that means that if you click on a link for a Cloudflare-supported site from DuckDuckGo, your referrer information does not reveal your search query or any personal information about you.)

Cloudflare’s report is updated quarterly, and the report can be split by operating system, device type, country, and month.

Wikipedia’s search market report

Wikipedia also recently published their search engine traffic data using a similar methodology. Every day Wikipedia counts link clicks from search engines and aggregates them into the search market share dashboard (also using direct referral data in a private manner).

We recommend Wikipedia’s data for more granular insights because their dashboard can be split in more ways, including by language, operating system, device type, and country, down to the day.

However, we recommend Cloudflare’s data to support higher-impact decisions because Wikipedia is just one site, whereas Cloudflare is based on millions of sites. While Wikipedia’s data is dependent on to what extent search engines include Wikipedia in their knowledge panels and in their search engine results, Cloudflare’s sample is so large that per-site effects are minimized.

In fact, we now believe Cloudflare’s report is by far the most accurate one of all search engine market share reports out there. With it, governments, researchers, and policy makers can better understand the search engine market and the effect of tools like search choice screens.