Use directory for default language causes problem with IndexNow

This is the technical support forum for WPML - the multilingual WordPress plugin.

Everyone can read, but only WPML clients can post here. WPML team is replying on the forum 6 days per week, 22 hours per day.

Sun	Mon	Tue	Wed	Thu	Fri	Sat
9:00 – 15:00	9:00 – 15:00	9:00 – 15:00	9:00 – 15:00	9:00 – 15:00	-	-
-	-	-	-	-	-	-

Supporter timezone: Asia/Yerevan (GMT+04:00)

Tagged: Bug

This topic contains 9 replies, has 2 voices.

Last updated by Christopher Amirian 4 days, 3 hours ago.

Assisted by: Christopher Amirian.

Author	Posts
October 27, 2024 at 8:13 am #16334063
mariuszM-13	Background of the issue: I am using IndexNow protocol to report to Bing all updates to pages and posts. On websites that are configured without the 'Use directory for default language' selected, links in both languages get properly reported to Bing. However, when the 'Use directory for default language' is selected, only the links in the first language are accepted by Bing. At Bing Webmaster Tools I have registered all three versions of the property: domainname.com, domainname.com/EN, domainname.com/PL. For reporting links with Index Now and Bing Webmaster tools I use the same Bing Webmaster Tools API key. For reporting links with Rank Math I use a separate key generated by Rank Math. This is not a key problem. The problem only exists when 'Use directory for default language' is activated. Symptoms: I get a 422 error on all second language links when the 'Use directory for default language' is selected, and only the first language links are accepted by Bing then. But when "Use directory for default language " is not selected, all links in both languages get properly reported. Questions: Why do I get a 422 error for second language links when 'Use directory for default language' is selected? How can I report links in both languages when 'Use directory for default language' is selected?
October 27, 2024 at 9:55 am #16334197
Christopher Amirian Supporter Languages: English (English ) Timezone: Asia/Yerevan (GMT+04:00)	Hi, The problem is reported. I will get back to you here as soon as I have news from second-tier support. Thanks.
October 31, 2024 at 1:59 pm #16351918
Christopher Amirian Supporter Languages: English (English ) Timezone: Asia/Yerevan (GMT+04:00)	Hi, I have an answer from the second-tier support. Please add the code below to the functions.php of your theme: function test_remove_homefilter() { if ( defined( 'REST_REQUEST' ) && REST_REQUEST && isset( $_SERVER['REQUEST_URI'] ) ) { $request_uri = $_SERVER['REQUEST_URI']; if ( strpos( $request_uri, 'submitUrl' ) !== false \|\| strpos( $request_uri, 'allSubmissions' ) !== false ) { global $wp_filter; if ( isset( $wp_filter['home_url']->callbacks[-10] ) ) { foreach ( $wp_filter['home_url']->callbacks[-10] as $key => $value ) { if ( str_contains( $key, 'home_url_filter' ) ) { remove_filter( 'home_url', $key, -10 ); } } } } } } add_action( 'rest_api_init', 'test_remove_homefilter' ); We tested that in the dummy website below and it worked: hidden link Thanks.
October 31, 2024 at 2:34 pm #16352217
mariuszM-13	Yes, this works with Bing Webmaster Tools, so I guess it will also work with Index now, as BWT is an extended version. However, there is still a problem with Rank Math Pro Instant Indexing. They also use Index now, but differently. So I guess a similar concept is needed for Rank Math.
October 31, 2024 at 3:49 pm #16352643
Christopher Amirian Supporter Languages: English (English ) Timezone: Asia/Yerevan (GMT+04:00)	Hi for that I will need you to tell me how to setup RankMath to have the Bing connection to be able to replicate the issue. I will report then to the compatibility team and then they can check how RankMath implemented the feature. Thanks.
October 31, 2024 at 4:01 pm #16352663
mariuszM-13	Instand Indexing is one of core functions of Rank Math. There is no need to setup anything there. Once you install Rank Math, it is there to use. They use a separate key, so you do not need to provide any key from Bing.
November 3, 2024 at 9:33 am #16357625
Christopher Amirian Supporter Languages: English (English ) Timezone: Asia/Yerevan (GMT+04:00)	Hi Thanks. The Rank Math plugin is installed on the sandbox installation below: hidden link I went to WordPress Dashboard > Rank Math SEO > Dashboard and found the "Instant Indexing" section and clicked the setting button there and I redirected to this page: hidden link There is an option that it is possible to add manual URLs and in the History section it shows the number of the HTTP response. I tested that and it is ok and is showing 200. The directory option is also enabled in WPML. So I am not sure how to replicate the issue as you described there. Thanks.
November 3, 2024 at 10:41 am #16357707
Christopher Amirian Supporter Languages: English (English ) Timezone: Asia/Yerevan (GMT+04:00)	Hi, It seems the code we suggested for a workaround also works for the RankMath SEO version. We published an erratum for that: https://wpml.org/errata/bing-url-indexing-fails-when-using-directory-for-default-language-with-wpml/ Thanks.
November 3, 2024 at 5:48 pm #16358311
mariuszM-13	I know why you cannot fully replicate the issue. Apart from the main Rank Math Pro plugin I also use and additional plugin caled Instant Indexing for Google also by Rank Math. https://wordpress.org/plugins/fast-indexing-api/ This plugin is only for google, but it integrates with Rank Math Instat Indexing and changes the layout of this section. Now, when I select two posts and from bulk actions I select to update in Google, it will open a modified Rank Math Instant Indexing page, where it presents me with a list of all links sent to google. Then all I need to do, is to select Index Now at the bottom and send the same links to Bing. And this is the part that does not work. It means that the lists of links are processed differently when sent to bing. I have installed the plugin on your sandbox wordpress. I haven't configured google indexing, as it is working properly anyway. And you do not need too worry about bulk actions either. So now all you need to test it is to go to Rank Math / Instant Indexing. Just like I did in the screenshot. In that section you will find a text area where you can put a list of links or just one link. If you use a second language link it will not work. But first language links do work.
November 4, 2024 at 9:56 am #16359729
Christopher Amirian Supporter Languages: English (English ) Timezone: Asia/Yerevan (GMT+04:00)	Hi, Thank you for the info. Just FYI, we do not have any integration with " Instant Indexing for Google" plugin of RankMath. I will report this to the compatibility team but I can not give a guarantee for a fix. Thank you for your understanding.