Resolved
Overview of the issue
Yoast SEO provides you a file editor by going to Yoast → Tools. This editor helps you create robots.txt files. WPML is not able to translate these files, which means the links will always point to the original domain.
Workaround
As workaround you can manually create several robots.txt files and place them in your WordPress site’s folder. For example:
- /robots-en.txt
- /robots-fr.txt
To do this, you first need to modify your .htaccess file and adapt this code with your site’s domains and robot filenames.
RewriteCond %{HTTP_HOST} ^www.domain.com [NC]
RewriteRule ^robots.txt /robots-en.txt [NC,L]
RewriteCond %{HTTP_HOST} ^www.domain.de [NC]
RewriteRule ^robots.txt /robots-de.txt [NC,L]
RewriteCond %{HTTP_HOST} ^www.domain.es [NC]
RewriteRule ^robots.txt /robots-de.txt [NC,L]
Then, to prevent WordPress from rendering a virtual robots.txt add:
touch robots.txt
Finally, create and edit the individual robots.txt files for each domain, using the file names as referred in .htaccess:
robots-en.txt robots-de.txt robots-de.txt