Background of the issue:
I am trying to automatically translate the website hidden link using WPML AI from Greek to English with a pay-as-you-go model. I have translated many posts and some custom post types, but some posts are not translating. I used Translation Management to add content under Λεξικό Διατροφής, selected 'Publish without review', and hit Translate. I also tried translating a post title 'Κοτόπουλο' via the advanced editor.
Symptoms:
Translations are stuck and not completing, even after waiting for 3 days. Cancelling and re-adding jobs charged me with translation credits without completing the translations. The post 'Κοτόπουλο' was refunded, but other posts were not.
Questions:
Why are my translations stuck and not completing?
Why am I being charged for translations that are not completed?
How can I resolve the issue of translations not completing in WPML?
Languages: English (English )Spanish (Español )Italian (Italiano )
Timezone: Europe/Rome (GMT+01:00)
Hello there,
Please watch this video: hidden link
We found some weird issues with non-latin characters and WPML AI which are already about to be fixed. when you see that happen you, can try again in a minute? it should work then. We'll release a patch with this fixed in a week or two, however this is an intermittent issue.
You can also send it to translate automatically so it works faster and has less risk of running into this (you can always edit it later if you want).
Since this was a problem on our end, I managed to get permission to reset your current credits usage 🙂
As I understand it, I should wait until you release your patch to use WPML AI, because until then it might be inconsistent and get stuck (and also charge me with credits without actually translating).
Since you are looking into this, I want to mention that the "word count" in the "translation management" page is wrong most of the time.
For example, in the one you were looking at "Χουρμάς", it says 4525 words whereas in reality it's almost half of that at about 2300-2500.
Would you mind checking that this is indeed the case? Maybe it's a bug for you to fix in future releases.
Languages: English (English )Spanish (Español )Italian (Italiano )
Timezone: Europe/Rome (GMT+01:00)
It's not that it's wrong, it's that we remove content or hide content that shouldn't be translated to make it cheaper and safer for you.
We get the info from the database and you have HTML code and other things that "count" as words but actually aren't supposed to be translatable ones.
At the same time in most of the situations you'll actually see less content translated because of factors like translation memory, content that might be turned into things like markers, content from theme or plugins options, etc.
The word count tool is what WordPress tell us you have in content and that's rarely more than the amount, but it can be less than that.
We're constantly trying to find ways to improve the count and remove from the count things that shouldn't be there but it's likely it won't be 100% exact because of the reasons mentioned above.
The good news is that you'll almost always end up spending less than what's shown in the word tool (which just to make clear is counting the WORDS, not the CREDITS needed).
-------------------
About the WPML AI. the issue won't happen every time, it usually tends to happen on pages that are a bit long and that also have non latin characters, so in your case it could be a bit more often than others. You can still use it, it won't happen everytime and since we're constantly pushing changes to the editor, it's likely that even from today you'll start encountering the situation less and less.
I understand, and indeed in the translation editor it seems from the credits that the words get calculated better.
However, before the latest WPML update, at the bottom of the "Translation Management" page there was a button "Word count for the entire site".
This would calculate the words pretty accurately in almost all the cases I have tried it, almost to the point. My suggestion is to use whatever code you were using to do that calculation in the new version as well, since it worked very well.
This is all just some friendly feedback since I use WPML for a lot of projects and would like to see it even more improved 🙂
Languages: English (English )Spanish (Español )Italian (Italiano )
Timezone: Europe/Rome (GMT+01:00)
I have already passed that feedback to our team and I have just one last question. how are you so sure it was quite accurate? how did you measure that?
This info is very important to us, it allows us to understand how you interact with this particular tool. I'm also asking this because I checked myself and found that the tool was quite accurate, however I believe that what makes the count "bigger" on our end are things like this:
Reference: Χουρμάς
Example Content:
"Al-Dashti," → splits into "Al" and "Dashti"
"Y. A.," → "Y" and "A"
"Holt," → "Holt"
"R. R.," → "R" and "R"
"Keen," → "Keen"
"C. L.," → "C" and "L"
"&" → becomes "amp"
"Hackman," → "Hackman"
"R. M." → "R" and "M"
"(2021)" → "2021"
That's more or less how the system would've seen some segments. the thing is that when it comes to these particularities it's not always as straighforward, it uses something like "weight" in words and it's not always as simple, but this is what makes it look way shorter to you and longer in the tool.
I checked other posts where these things weren't added and the count was spot on.
You also have things like Yoast SEO excerpts, WP Recipe excerpts, a nutrional terms value or something like that almost at the end of the page, all those things count in the word count.
I'd Greatly appreciate your view on this and how do you find out about the word count.
All the posts we have are first composed on a Google doc where we can see the total count of the words. I don't have the doc for Χουρμάς because it's old, but if you paste the contents in a new doc it counts them up to 2298.
hidden link
Also I have the following sql query which will give a rough estimate of the words of a post, however depending on if the post has html/shortcodes/other tags it might be off.
SELECT SUM(LENGTH(post_content) - LENGTH(REPLACE(post_content, ' ', '')) + 1) AS total_word_count FROM wp_posts WHERE ID = 1444;
For Χουρμάς, it counted them up to 2657.
In the Translation Management page, it estimates the words for Χουρμάς to 4525, which you can tell is far off from both the other methods, even accounting for things like yoast seo tags, taxonomy names etc
Before translating this website, I run this SQL code for all posts and got an estimate of 800k-900k words. With the previous version that had the "Word count for the entire site" button, it would calculate really close to that. Using the translation manager in this version and selecting all content, it estimates it at more than double the words (I think something like 1.7mil).
Languages: English (English )Spanish (Español )Italian (Italiano )
Timezone: Europe/Rome (GMT+01:00)
I already sent feedback to our devs but I tried to test it myself and I have to say that unless you remove exactly all the HTML (including the lots of attributes that are added to your pages) then the amount will never even be close to what you mentioned (2.5k words).
hidden link
Our devs will look into this, of course but I also tried something else. copied the content without the table and turns out that's why you see 2.5k more or less instead of 4.5k : hidden link
I also tested in a gutenberg editor by pasting only the content, without the table and that amounts to your count. However the table contains so much stuff that we're actually filtering (That's why it's 4.5 and not 4.6 or more) but only the things that are SAFE to filter. some others should be filtered out when we see this specific scenario but they aren't because they "could" also be part of things that should be translatable.
Anyways, this was just an extra step to explain our process, but your feedback is duly noted and we'll definitely see how we can improve this tool even further (this is the fifth iteration of the tool and in our tests it was showing like a 4-5% improvement on the reliability). Unfortunately it will never be 100% precise for the things i mentioned in the past.
I understand, there is html code that doesn't get filtered in the word calculation function because there might be things that need to be translated.
This seems indeed to be the case, as some of my other pages have a ton of WP Bakery code, but the word count for them is correct.
Maybe my scenario with the table that seems to not be filtered, includes something "interesting" that might help your team improve the word calculation function.
It just so happens that a lot of my pages contain tables, and so the word calculation for those is incorrect due to this issue.
Hopefully I helped in some way, if you need anything else I will be glad to assist!
And thank you for the excellent support in this whole thread.
Languages: English (English )Spanish (Español )Italian (Italiano )
Timezone: Europe/Rome (GMT+01:00)
Yes, this scenario is a nice case study which our devs are already looking into 🙂 it's definitely to know about these cases because they help us improve so for that I thank you again for bringing it up.
I'll leave the ticket open for 14 extra days in case you need any further assistance from me.
