SEO Strategies That Use Natural Language Processing

NLP is, essentially, an implementation of AI (artificial intelligence) that allows search engines (and your business) to extract meaning from user-generated text, including that on your website. As such, it represents one of the key ways that AI is changing SEO, alongside the rise of voice search and the figurative explosion in the use of chatbots over the past few years.

NLP remains, however, little understood by SEO specialists and business owners alike. This article will delve into and explain (hopefully clearly and succinctly) how NLP works and how best to deploy it to improve your SEO.

To get up to speed on the development of NLP and its fascinating history in short order, this article published by Sean Shuter a few years ago provides insight into the intersection of NLP and SEO in understandable language.

To understand just how vital NLP will be to the future of SEO, it's worth looking at how Google – in particular – implements NLP and just how central it is to their mission.

One of the issues that researchers have faced when it comes to convincing businesses of NLP's value is that the idea of using these models to read real-life text is inherently dated. As Dixon Jones, CEO of Inlinks.net, recently put it, though, this is a huge mistake: "When people realize NLP stands for Natural Language Processing instead of some 1970s hypno-mumbo-jumbo," he said, "they'll realize that not only is it here to stay, it's the very bedrock of the mantra organizing the world's information."

Google's NLP approach is built on a ground-breaking language processing model: BERT (Bidirectional Encoder Representations from Transformers). BERT is outlined in a recent paper published by researchers at Google AI Language. Its publication caused quite a stir in the Machine Learning community by presenting state-of-the-art results in a wide variety of NLP tasks, including Question Answering (SQuAD v1.1), Natural Language Inference (MNLI), and others.

BERT's key technical innovation is applying the bidirectional training of Transformer, a popular attention model, to language modeling.

While that might seem technical, BERT's impact on SEO can be put pretty simply: Google no longer looks at words or phrases individually, as we understood in the past when it would traditionally run keyword research. Now it looks at sentences, paragraphs, and the query as a whole. In other words, the algorithm looks at the sentiment or overall intent rather than focusing on individual words.

Google's official Search Liason recently revealed on Twitter that Google BERT is now helping with one out of every ten google searches in the US in English, with plans to expand soon to include searches in more countries and languages.

For Google, all of this research and investment in NLP is geared toward one outcome: improved search quality. Just as it always has, this search engine juggernaut continues to push for a better way to provide the exact information that users are after.

There are many challenges in this pursuit. According to the information on Google's blog, 15% of search queries are used for the first time. In other words, people are using more long-tail searches to find an answer to their question, especially with the rise of voice search.

Another is that computers – even those equipped with advanced AI systems – are still not that good at assessing the relevance and authority of information on web pages. This capability is increasing all the time, of course, and it might well be that in 2030 we regard the NLP capabilities of today as primitive. However, the fact remains that the SERP for most pages is still more a product of the keywords it contains than its genuine utility to most users.

NLP is an attempt, on Google's behalf, to change that. According to recent tweets from the Stanford NLP Group, NLP is now a key component of AI and integral to SEO practices.

And tweets from AI expert Bernard Marr affirm the power of NLP to draw on our widespread use of natural human language. One day, of course, we might be in a position where great quality content, irrespective of the SEO techniques applied to it, ranks well on its own. But until that stage, we need to rely on tools to highlight our pages for BERT.

A second way in which NLP is currently impacting SEO is in internal linking structures and content recommendation tools.

These two functions may seem fairly distinct, but a closer look at how NLP engines work reveals that they are inherently (pardon the pun) linked. When an NLP system encounters a term that it doesn't understand, it attempts to resolve its meaning. If it can do so by referring to the material on your site, it grants you an SEO boost in recognition of this fact. To see this in action, take a look at how The Guardian uses it in articles, where the names of individuals are linked to pages that contain all the information on the website related to them. Robert Weissgraeber, CTO of AX Semantics, notes that NLP boosts brand visibility with no additional effort by creating huge quantities of natural language content.

The same type of process underpins more advanced forms of content recommendation. There are now several content optimization tools available that can assess the meaning of the content that appears on a user's list of visited sites and takes advantage of this to suggest other content. While some of this functionality can be limited by browser privacy modifications, it provides a powerful way for websites to increase the "dwell time" on their sites.

Finally, more speculative use of NLP for SEO. In recent years, we've seen a shift from marketers targeting individual keywords to targeting topic clusters. As search engines become more capable of understanding the world around us and disambiguation comes into play, the same results can be presented to the user across multiple searches that share the same intent; the competition is no longer on targeting a specific keyword but rather seeks to become relevant for a specific topic. Experts note that NLP can be used to identify patterns, providing search results along a theme.

In other words, it could be that in the near future, SEO will revolve more around providing high-quality, human-readable content, rather than the keyword-driven content of today.

Though NLP might not be on your radar just yet, you should be aware that it is one of the key trends of 2020 and that it will almost certainly have a widespread impact on SEO within the next few years. It would be a good idea to familiarize yourself with its capabilities. A fun and instructive way to do just that is to look at Google's Natural Language API demo, allowing you to examine any text for free.