Robots.txt

What is Robot.txt ?

It is the file in which we give instruction to Google crawlers (Internet programmers / Google bots/Google spiders) which page has to crawl or not Where we just use some Disallow and Allow the code.

How to create Robots.txt ?

Go to WordPress > Tools > plugin > Yoast SEO Plugin > File Editor > Create Robots.txt file

Than apply changes according to requirements

How to check Robots.txt working properly ?

we search Https.//www.ourdomain.subdomain/robots.txt

in browser's search bar.

Example : 

Https://www.gitmasterg.online/robots.txt

in our browser's search bar.