List of Googlebot IP addresses released.

Google has published a complete list of IP addresses that Googlebot uses as user agents to crawl and access your website. Google said that if you do not want to use other methods of reverse DNS to authenticate Google bot, you can use “IP address” to give Google bot the IP address of the crawler and the list of Google bot IP addresses. Can identify with. “

Why do I need it? If you believe that someone is crawling your site, pretending to be a Google bot, and you want to confirm that blocking this fake Googlebot crawler is not really Google, then you or So you can use the command line method or the automatic IP list method.

IP list. You can access the full list of Googlebot IP addresses in this JSON file here, note that Google may update this list, so you should probably check the file daily. You can also access all the other Google crawlers that match the IP address of the crawler compared to the full list of Google IP addresses. Again, these IP addresses may change over time, so you should work out ways to update the IP addresses in your procedure.

Why do we care Most sites can be slowed down and possibly even offline because fake bots are crawling and spying on the site. You rarely want to prevent Google from crawling your site as this can lead to indexing and ranking issues in Google search. So knowing which bots are really Google and which are not, can help you decide which malicious bots to block from your site.

There are also third party services, such as Cloudflare and others that help you manage this.

About the Author

Barry Schwartz is the Assistant Editor of Search Engine Land and a member of the programming team for SMX Events. He owns RustyBrick, a web consulting firm based in NY. He also runs the Search Engine Roundtable, a popular search blog on highly advanced SEM topics. Berry’s personal blog is called Cartoon Berry and can be followed on Twitter.

Write a Comment

Your email address will not be published. Required fields are marked *