Let's assume that there is a page like example.com . In the root, there is a robots.txt file. Let's also assume that we have the following structure for different pages for different languages: example.com/UK example.com/DE etc. The crawler always runs into 404 when it tries to crawl the example.com/de/robots.txt Is it possible to tell the crawler to search in the root?

Robots.txt is not found in subdirectories

Let's assume that there is a page like example.com. In the root, there is a robots.txt file. Let's also assume that we have the following structure for different pages for different languages:

example.com/UK
example.com/DE
etc.

The crawler always runs into 404 when it tries to crawl the example.com/de/robots.txt

Is it possible to tell the crawler to search in the root?

Robots.txt is not found in subdirectories

0 Answers