Google’s non-existent whitelists… exist


Icon from the Tango Desktop Project

Google has been caught manipulating search results again, but at least they admitted what they were doing!

Google mostly do no evil :)

Along with IPv6 and cloud computing in their current forms, I've viewed most of my concerns with Google in light of potential government and law enforcement abuses. Whereas even more privacy and security obsessed people than me have been quick to dismiss Google as hypocritically evil, I've been willing to give them the benefit of the doubt.

Take the street view controversy. While I think Google engineers were short sighted by not closely studying the source code of the software they put on their trucks and drove around the world, I don't believe they did it maliciously. These signals were being broadcast in the open, and while the scale of Google's downloading may warrant further scrutiny, it skips the real issue that people are still broadcasting unencrypted data out of their homes for anyone to gain access to. Rather than blaming consumers (which is always an easy thing to do) however, I place the blame on network hardware manufacturers for selling devices that didn't make this clearer.

Basically, while I've taken issue with some of their policies (labelling Android as "open" and broadly copying the iOS interface, the Verizon net neutrality thing, Eric Schmidt's baffling statements about not having things to hide, pushing WebM and Flash, claiming redundancy is built into Gmail only to have all that redundancy fail), I still believe they're a force for good online and we benefit from their presence. Clearly, Bing does too ;).


Unfortunately, it's come to light that Google has been misrepresenting aspects of their search algorithm's operation. After steadfastly and repeatedly denying in court that they use whitelists to inform their search results, both Google and Bing have admitted their existence. These so-called "exception lists" are claimed to consist of sites that Google and Bing engineers deem to have been false positives in their farmer site targeting algorithms.

As with all stories of this nature, how its being reported is almost as fascinating as the story itself. Barry Schwartz at SearchEngineLand catches out Google for denying whitelists are used in their anti farming efforts but otherwise sticks to reporting the facts.

Matt Cutts explained that there is no global whitelist but for some algorithms that have a negative impact on a site in Google’s search results, Google may make an exception for individual sites. But didn’t Matt tell us there is no whitelist for the Farmer update?

Cade Mets at The Register was less charitable and accused Google of lying to European Union anti-trust regulators.

The trouble is that Google spent years refusing to acknowledge that these manual interventions exist – and, in some cases, outright denying them. Google spent nearly a decade telling the world that its search engine was completely objective, and it has only recently begun to freely admit that this is not the case, presumably as a result of the EU investigation.

Ruben gets nervous talking to people

I've talked to people at university and my old work about this, and as usual the opinions are pretty evenly divided between "Google is guilty of denying the true nature of their operations and must be punished", to "Google's exception lists are a valid approach to dealing with false positives, these stories are just sensationalist".

Sensationalist sounds like it'd be a good thing. What a sensation! How sensational! YAYS! The English language is weird.

I'm of the opinion that while Google have been guilty of search result rigging in the past, we don't have enough evidence yet to suggest Google are abusing these exception lists. If they are as benign as they suggest, I'm sure Google shouldn't have any problem with providing us with further details as to their operation. If they remain obtuse, then we have severe cause for concern.

Author bio and support


Ruben Schade is a technical writer and infrastructure architect in Sydney, Australia who refers to himself in the third person. Hi!

The site is powered by Hugo, FreeBSD, and OpenZFS on OrionVM, everyone’s favourite bespoke cloud infrastructure provider.

If you found this post helpful or entertaining, you can shout me a coffee or send a comment. Thanks ☺️.