Should I add a Robot.txt file for Blogger blogs in 2023?

Should I add a Robot.txt file for Blogger blogs in 2023?

 Many bloggers and YouTubers advise their followers to create bot files for their blogs, and each of them claims that his file is the best, so do I really need to add the bot file to my Blogger blog?

And the answer is, of course, no! Unless you want to hide specific links within your blog, you don't need to add a Robots.txt file to it.

What should I do, then? Don't change anything, just leave the tags corresponding to the bot file in the settings as they are (in grey) and the blogger will create a bot file suitable for your blog.

User-agent: Mediapartners-Google

Disallow: 

User-agent: *

Disallow: /search

Allow: /

Sitemap: https://yourdomain.com/sitemap.xml

Put your blog address in place of https://yourdomain.com and see what happens; you'll find a bot file that exists even though you haven't added any files to your blog; this is the best bot file, especially for new blogs.

Explanation of the default bot file for blogger

  • Let us explain the meaning of the commands in this file for the sake of knowledge now that we know what the default bot file for blogger is and that no file should be added.
  • Each command in the bot file has two parts: user-agent followed by a specific name or *, this line specifies the name of the search engine program or bot, and if the name is *, the command includes all programs.
  • The second component is a set of commands like Allow or Disallow that tell the search engine what to archive and ignore.


User-agent: Mediapartners-Google 

Disallow:

  • It entails granting the Google AdSense ad crawler access to all of your website's pages.

User-agent: *

Disallow: /search

Allow: /

  • It means allowing all search engines to index your site, excluding search and label pages, and allowing the entire site to be indexed on the third line.
  • The final section specifies the type of sitemap that can be indexed.
  • We hope this article cleared up any confusion about where to find the blogger's robot file; if you want to learn more, go to the next page of the official Google website.

Next Post Previous Post
No Comment
Add Comment
comment url