Sitemap Generator Forum

Should the robots.txt be empty? How can we manually add information to robots.txt? I do not see an option for adding code anywhere for the robots file. I know you can add or disallow URLs but where can we actually add info to robot file?

Hello,

empty robots.txt allows all pages to be indexed (which is ok).
If you want to exclude a folder, for instance, you can add this:
User-agent: *
Disallow: /foldername/

where do you add these lines?

In robots.txt file (create a new file if you don't have one yet)