ATE translation issues

This thread is resolved. Here is a description of the problem and solution.

Problem:
The client was facing an issue with the Advanced Translation Editor (ATE) where certain items on their menu page were not being translated correctly, specifically those containing emojis. Initially, replacing emojis with text seemed to resolve the issue, but the client wanted a solution that allowed the use of emojis.
Solution:
We investigated the issue and found that it was related to the database collation settings. The database was using the

utf8_general_ci

collation, which does not fully support emojis as they require 4 bytes per character. We recommended changing the database collation to utf8mb4_unicode_ci or utf8mb4_general_ci to properly support emojis. After the client converted their database to

utf8mb4_unicode_ci

, the issue was resolved, and emojis were displayed correctly in translations.

If you're experiencing a similar issue, we recommend checking your database collation and updating it to utf8mb4_unicode_ci to ensure full support for emojis. However, this solution might be irrelevant if it's outdated or not applicable to your case. If the issue persists, please open a new support ticket. We also highly recommend checking related known issues at https://wpml.org/known-issues/, verifying the version of the permanent fix, and confirming that you have installed the latest versions of themes and plugins.

This is the technical support forum for WPML - the multilingual WordPress plugin.

Everyone can read, but only WPML clients can post here. WPML team is replying on the forum 6 days per week, 22 hours per day.

Our next available supporter will start replying to tickets in about 7.41 hours from now. Thank you for your understanding.

Tagged: ATE

This topic contains 14 replies, has 2 voices.

Last updated by Bobby 2 months, 2 weeks ago.

Assisted by: Bobby.

Author	Posts
January 25, 2024 at 12:44 pm
John-Pierre Cornelissen	Hello, I have an urgent issue with the ATE. It's regarding the English and German translation of this page: hidden link If you look at the translations, you see that a lot of items have not been translated. However, when I go to review, everything is properly translated.
January 25, 2024 at 1:28 pm
January 25, 2024 at 2:36 pm #15226773
John-Pierre Cornelissen	Hello, any news on this? I need this translation to be delivered to the customer today. Thanks JP
January 26, 2024 at 12:29 am #15228511
Bobby Supporter Languages: English (English ) Timezone: America/Los_Angeles (GMT-07:00)	Hi there, I have tested the issue on your site and I am discussing this with the appropriate team at the moment. Once there is news we will update you here. In the meantime if it is crucial to have this page 100% translated a workaround would be to manually add the translation by going to the page, viewing the translation in the front end, and clicking on "edit with visual builder" at the top header, this will allow to add the translations manually.
January 26, 2024 at 5:48 pm #15231345
John-Pierre Cornelissen	Thank you for the update yesterday Bobby. Did you find the cause of the issue? And what did the team say about fixing it? Thanks JP
January 26, 2024 at 7:09 pm #15231591
Bobby Supporter Languages: English (English ) Timezone: America/Los_Angeles (GMT-07:00)	Hi JP, we are still investigating this issue, however, I did come to a finding: I created a new page called WPML Test and added a row with 3 text elements. In the first 2 I copied the first 2 menu areas which are not tranlsating 100%, then for the 3rd I copied one from the ones that are tranlsating OK. I got the same results as the original menu page. With the 4th text element instead of copying the text, I added it by typing it in and it seems to work OK. (See screenshot) It is almost as if the text element and it's content are corrupted. In the meantime, our team is still reviewing this for a more definite answer, but please do give it a try on the sample page I created and let me know your results. P.S The page is called WPML Test and marked as private
January 26, 2024 at 9:24 pm #15231727
John-Pierre Cornelissen	Hi, I think I know what might be the problem. I can't test myself at the moment as I am on the road and on my phone right now, but it looks like the parts that are not translating all have an emoji in them (to indicate a vegetarian dish). Could that be the problem? JP
January 27, 2024 at 5:10 pm #15233151
John-Pierre Cornelissen	Hi, I can confirm now that it is caused by the Emoji character. I replaced them with (V) and the translations are correctly being picked up now. I would like to see a better solution though, so the emoji's can be added back again. Thanks JP
January 30, 2024 at 1:07 am #15241309
Bobby Supporter Languages: English (English ) Timezone: America/Los_Angeles (GMT-07:00)	Hi JP, Thank you for confirming this is caused by the emojis added. In the backend, where and how is the emoji added? Is it added directly in the text editor? When I was testing I was not able to see it or perhaps I missed it.
January 31, 2024 at 9:41 am #15248265
John-Pierre Cornelissen	Hi, Yes they are added in the text editor. On Windows 11, you can open the emoji selection popup when you press the windows key + the . (dot) at the same time. Have a look at your test page: hidden link The php version seems to be of influence, but I haven't exactly figured out yet how. This site is on php 8.1. I tried to replicate on my wpml test site, but I get other issues there when adding emoji's (even on the source language). Some guidance would be appreciated. Thanks JP
January 31, 2024 at 6:48 pm #15251293
Bobby Supporter Languages: English (English ) Timezone: America/Los_Angeles (GMT-07:00)	Hi JP, Thank you for updating us! I can see the emoji in the link you shared with me. I spoke with our team and what we found is that If the problem is the emoji, this means that the db table is not with collation utf8mb4_unicode_520_ci and should be adjusted to that collation. We reviewed the collation and site-wide it's using utf8_general_ci with database collation latin1_swedish_ci You can store emojis in a MySQL database table field but with some important considerations regarding the collation. The utf8_general_ci collation in MySQL does not fully support emojis because it is based on the UTF-8 encoding, which supports only a subset of Unicode characters (up to 3 bytes per character). Most emojis require 4 bytes to be represented, as they are part of the supplementary character set in Unicode. To properly store emojis, you should use the utf8mb4 character set, which supports four bytes per character and can accommodate all Unicode characters, including emojis. The corresponding collation would typically be utf8mb4_unicode_ci or utf8mb4_general_ci. Additionally, I did take a look on the WordPress forum and did find a few users mentioned the PHP change triggered some issues, however, that was related to the fact that when the PHP version was changed some modules were changed causing their issues. https://wordpress.org/support/topic/php-8-doesnt-support-emojis/ Let me know your results, please.
February 1, 2024 at 4:48 pm #15256068
John-Pierre Cornelissen	Thank you for investigating and the insight Bobby. I'll check the php extensions and see if I can convert the database. Will get back to you after the weekend.
February 5, 2024 at 11:35 am #15265715
John-Pierre Cornelissen	I will get back to this, pls don't close the ticket.
February 5, 2024 at 7:37 pm #15268236
Bobby Supporter Languages: English (English ) Timezone: America/Los_Angeles (GMT-07:00)	Hi Jp, Thank you for updating me, the system has an automatic closing procedure, however this should reset it for now. If for any chance it does close, please open a new ticket and request that it is assigned to myself so we can continue
February 7, 2024 at 6:36 pm #15278622
John-Pierre Cornelissen	I know but I prefer to keep it open and have everything in this ticket.

John-Pierre Cornelissen confirmed that the issue was resolved on 2024-02-09 11:47:23.
This ticket is now closed. If you're a WPML client and need related help, please open a new support ticket.