I'd like to use the chrome audits tool on my React app, but it keeps saying that my robots.txt file is invalid. The thing is, the file seems to be perfectly fine, only it doesn't read robots.txt, but rather my index.html file, therefore resulting in this error :
Both files are in my /public folder, so why would it read the index file ?
if your using Node and Express in order to run your React App project in production. and if your server file code looks like below snippet, this means your are serving same file for all requests(routes). During lighthouse auditing, browser makes a get request to http://localhost:5000/robots.txt and in return it get served with index.html.
app.get('/*', function (req, res) {
res.sendFile(path.join(__dirname, 'build', 'index.html'));
});
To solve this problem ,you can add route for robots.txt file, before above code snippet as shown below.
app.get('/robots.txt', function (req, res) {
res.sendFile(path.join(__dirname, 'build', 'robots.txt'));
});
app.get('/*', function (req, res) {
res.sendFile(path.join(__dirname, 'build', 'index.html'));
});