サポートフォーラム アーカイブ

Problem:
The client has a multilingual site with Spanish as the root language and English as an additional language. They have configured a robots.txt file for the Spanish version, which includes SEO configurations for both languages. However, they noticed that the English version (/en/) has a robots.txt file without any configurations and lacks a sitemap file. The client is concerned about the potential SEO impact.
Solution:
We confirmed that search engines do not look for robots.txt files in language-specific folders, so this will not impact SEO. However, to address the issue of the unconfigured robots.txt file in the English directory, we provided a temporary workaround:
1) Ensure you have a full site backup.
2) Open the

functions.php

file in your Child theme for editing.
3) Add the following code to the bottom of the file:

add_filter( 'rewrite_rules_array', 'remove_robots_rewrite_rules' );

function remove_robots_rewrite_rules( $rules ) { 
unset( $rules['robots\.txt$'] );
return $rules;
}

4) Save the changes.
5) Go to Settings >> Permalinks and resave the settings.
We are currently working on a permanent fix and will update the client as soon as it is available.

If this solution does not resolve your issue, or if it seems outdated or irrelevant to your case, we highly recommend checking related known issues at https://wpml.org/known-issues/, verifying the version of the permanent fix, and confirming that you have installed the latest versions of themes and plugins. If further assistance is needed, please open a new support ticket at WPML support forum.