Back in 2016 Nick Waddell uploaded a work of Ernest Hemingway into his WordPress install to see how the Yoast SEO plugin would react:
After months of seeing this needling assessment I decided to try an experiment. I would find a collection of words generally accepted to be masterpiece -a no-doubt about it piece of literature- and load it into Yoast and see what happened. Ernest Hemingway’s “The Snows of Kilimanjaro” seemed to fit the bill. For those who don’t know, “The Snows of Kilimanjaro” was first published in Esquire magazine in 1936 and generally regarded as one of Hemingway’s masterpieces, alongside “”The Sun Also Rises” and “For Whom the Bell Tolls”. I posted the entire 9000 plus word story as though it were a new Cantech Letter article and hit “Save Draft”.
[Yoast’s content score:] “Needs improvement”.
Nick’s assessment was that the SEO plugin was grading the words against the Flesch Reading Ease score, which was developed in the 1970s by the US Navy to improve the readability of technical documentation. So it’s understandable why a highly-regarded work of literature would score poorly.
One may argue technical sites could benefit from using this as tool, which presumably informed Yoast’s inclusion of the metric in its plugin by default. We nerds are nothing if not technocratic: if we can mechanically turk a solution to a human problem, why not let a computer help?
Where I worry is the predictably regular misapplication of these types of metrics into fields for which they were never designed, under a deliberate or misguided idea that it must improve everything. When it’s applied to art we get worse art; just look what autotune did to a generation of music, or the drive for clickbait did to journalism.
Nick Waddell’s Cantech Letter site is a long-form science journal targeting those interested in medicine and research. Should this be held to the same score requirements as a general-purpose encyclopedia, or the FreeBSD Handbook, or other docs? Extrapolating this out further, are we okay with this plugin telling all authors that their content “needs improvement”, from bloggers to playwrights? For those who aren’t as technically inclined, would they even be aware that it’s an SEO plugin telling them this? I can’t imagine the negative reinforcement seeing that terse, negative assessment every day would have.
I wrote about a different online tool struggling to correct grammar back in 2018. How many other articles, posts, books, journals, and other literary works have been mangled or changed by such algorithms? Maybe I’m overthinking this as I’m want to do, but I’d still guarantee you that number is more than zero.
I’m glad that we’re all waking up about inherent biases designed and trained into our algorithms, from the finance sector to the job market. But I think just as fundamental is the question of whether an algorithm should even be employed in certain cases, especially when directing art. At best it provides marginally-helpful advice, at worst it attacks it.