Content Marketing, Featured, SEO

What Is BERT? Does it affect your SEO Strategy?

May 28, 2020 | 3 minutes to read
Summary:Google regularly rolls out updates to its search algorithms. Google rolled out BERT during the week of October …

Google regularly rolls out updates to its search algorithms. Google rolled out BERT during the week of October 21, 2019. The changes related to BERT impact Google’s search capabilities in over 70 languages.


What is BERT?

BERT (Bidirectional Encoder Representations from Transformers) was a paper published in 2018 by Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova from Google. It is a neural network-based technique for natural language processor pre-training. Google has leveraged BERT’s capabilities in its recent algorithm updates!

Algorithmic changes in the past have significantly impacted SEO rankings and marketers’ efforts to stay current with changed guidelines to maintain their Google search rankings. In that sense, BERT may be an exception in that it may need a little to no change in the direction of the marketers to stay on top of the Google search. Surprised?, read on to get more details of BERT and how this can affect your SEO strategy or not!

BERT has created a buzz in the Machine Learning (ML) community by presenting very encouraging results in a range of NLP tasks, including Natural Language Inference, Question Answering, and such.

How is BERT different from previous Natural Language Processing (NLP) algorithms?

The technique employed in BERT is a significant departure from previous efforts. Here, the transformer encoder reads the entire sequence of words at once, instead of left to right or combined left-to-right and right-to-left training.

BERT employs MLM and NSP training strategies in predicting the next word in sequence.

Masked Language Model (MLM)

Masked language modeling is a fill-in-the-blank task, where a model uses the context surrounding a [MASK] token to try to guess what the [MASK] should be. 

Next Sentence Prediction (NSP)

To understand the relationship between the two sentences, the BERT training process uses the next sentence prediction. BERT model gets input pairs of sentences, and it learns to predict if the second sentence is the next in the original text as well.

What is new with BERT? 

Old and conventional language models use only previous tokens to predict the next token, whereas BERT uses both the previous and the next tokens. Bidirectional refers to BERT using both the left and right sides of a token’s context during the training.

Why does BERT matter to SEOs?

Search intent is a significant factor in Google’s search results. BERT is designed to better understand the “intent” behind a user’s query compared to Google’s previous versions of the algorithms. Google has said that 10% of all searches are impacted by the BERT update.

Can you expect different search results from BERT?

Yes, in some cases!

Pre-BERT Google search returned the same SERP for “Parking on a hill with no curb” and “Parking on a hill.” The word “no” is an essential part of the search phrase. The pre-BERT algorithm missed that part!

BERT knows grammar! The pre-BERT algorithm had a severe limitation in interpreting the prepositions and pronouns. The search phrases like “over the moon” and “on the moon” returned the same results.

Let’s say you Google the phrase “2019 Brazil traveler to USA need visa”.

In the Pre-BERT search, on the U.S. Google page, it would bring about a page telling whether American citizens need a visa to go to Brazil.

But that’s not the intent of the query. The word “to” is a significant token indicating “people traveling to the USA” pre-BERT algorithms ignored this. With BERT, the top result is the U.S. Embassy in Brazil, which shows Brazilian nationals who want to go “to the USA” to apply for the relevant visa required.

Do you have questions about Google updates and how it affects your Digital Marketing strategy? We will be happy to discuss this. Please set up an appointment today!


Do you need to Optimize for BERT?

“There’s nothing to optimize for with BERT, nor anything for anyone to be rethinking. The fundamentals of us seeking to reward great content remain unchanged.” — Danny Sullivan, Google (@dannysullivan) October 28, 2019

So, Google emphasizes that it’s going to continue rewarding the great content that helps users. So, keep producing useful and great content!

In summary, according to Google, nearly 10% of all searches are impacted by BERT. Long-tail keywords now return more meaningful results compared to the pre-BERT algorithm because BERT can better interpret the search intent. BERT will also impact featured snippets.





About the Author

Krishna Murthy, Marketing Manager at WSI, is a Digital Growth Consultant, Photographer, Travel Enthusiast, Software Engineer who is continually working on building a better marketing campaign for clients!

The Best Digital Marketing Insight and Advice

The WSI Digital Marketing Blog is your go-to-place to get tips, tricks and best practices on all things digital
marketing related. Check out our latest posts.

    I consent to WSI collecting my contact details and sending me digital communications.

    We are committed to protecting your privacy. For more info, please review our Privacy and Cookie Policies. You may unsubscribe at any time.

    Don't stop the learning now!

    Here are some other blog posts you may be interested in.VIEW ALL BLOG POSTS

    Unlock the Potential of Generative AI for Your Business: Benefits and Concerns

    February 22, 2023 | 6 minutes to read

    What is Generative AI? In an ever-evolving technological landscape, companies are increasingly looking to artificial intelligence (AI) to …


    ChatGPT: How can businesses adopt this AI model?

    December 15, 2022 | < 1 minutes to read

    ChatGPT is a state-of-the-art language generation model developed by OpenAI. It is capable of generating human-like text based …


    How can SEM Profit Law Firms in this Economy?

    April 22, 2022 | 4 minutes to read

    Search Engine Marketing (SEM) Simply put, Search Engine Marketing is where a business uses search engines such as …


    © 2022 WSI Kriti Web Solutions, LLC. All rights reserved. WSI ICE and WSI IM are registered trademarks of RAM. Privacy Policy and Cookie Policy. Each WSI Franchise is an independently owned and operated business.