<b>Background of the issue: </b>
I am trying to sync a large number of translation jobs on my site, hidden link, using Crowdin's API. We have a global scope and a very large site, with approximately 250,000 translation jobs to do. I've performed AI translation on them using Crowdin's API, and some have popped up, but mostly they all seem to be stuck now. Previously, I used ATE and switched to CTE, restarting completely fresh, but encountered issues once there were more translation jobs. I've tried pressing the no/very-low risk troubleshooting buttons. Some database queries seem to be taking a long time. Here's the Crowdin project: hidden link Here are some of the queries: require('wp-admin/menu.php'), wp_count_comments, get_comment_count, get_comments, WP_Comment_Query->query, WP_Comment_Query->get_comments, WP_Comment_Query->get_comment_ids, QM_DB->query #121 (52.2 ms) SELECT COUNT(*) FROM wp_comments JOIN wp_icl_translations icltr2 ON icltr2.element_id = wp_comments.comment_post_ID JOIN wp_posts ON wp_posts.ID = icltr2.element_id AND CONCAT('post_', wp_posts.post_type) = icltr2.element_type LEFT JOIN wp_icl_translations icltr_comment ON icltr_comment.element_id = wp_comments.comment_ID AND icltr_comment.element_type = 'comment' WHERE ( comment_approved = '0' ) AND icltr2.language_code = 'en' AND (icltr_comment.language_code IS NULL OR icltr_comment.language_code = icltr2.language_code) do_action('load-toplevel_page_tm/menu/main'), WP_Hook->do_action, WP_Hook->apply_filters, WPMLUserInterfaceWebInfrastructureWordPressCompositionRootConfigAdminPage->WPMLUserInterfaceWebInfrastructureWordPressCompositionRootConfig{closure}, WPMLUserInterfaceWebInfrastructureCompositionRootConfigConfig->onLoadPage, WPMLUserInterfaceWebInfrastructureCompositionRootConfigConfig->loadScripts, WPMLUserInterfaceWebCoreComponentDashboardApplicationDashboardController->initialScriptData, WPMLInfrastructureWordPressComponentTranslationProxyApplicationQueryRemoteJobsQuery->getCount, WPMLInfrastructureWordPressPortPersistenceQueryHandler->querySingle, QM_DB->query #183 (136.3 ms) SELECT COUNT(*) FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON t.term_id = tt.term_id LEFT JOIN wp_icl_translations icl_t ON icl_t.element_id = tt.term_taxonomy_id AND icl_t.element_type IN ('tax_category') WHERE tt.taxonomy IN ('category') AND ( ( icl_t.element_type IN ('tax_category') AND ( icl_t.language_code = 'en' OR 0 ) ) OR icl_t.element_type NOT IN ('tax_category') OR icl_t.element_type IS NULL ) do_action('toplevel_page_tm/menu/main'), WP_Hook->do_action, WP_Hook->apply_filters, WPMLUserInterfaceWebCoreSharedKernelConfigPage->render, WPMLUserInterfaceWebCoreComponentDashboardApplicationDashboardController->render, WPMLUserInterfaceWebLegacyComponentDashboardDashboardTabs->wrapTabsAroundContent, WPML_TM_Menus_Management->renderEmbeddedDashboard, WPML_TM_Menus_Management->render_main, WPML_TM_Menus_Management->build_tab_items, WPML_TM_Menus_Management->build_translation_jobs_item, WPML_TM_Jobs_Repository->get_count, QM_DB->query #219 (820.2 ms) SELECT wpml_translations.translation_id, wpml_translations.element_id, wpml_translations.language_code, wpml_translations.source_language_code, wpml_translations.trid, wpml_translations.element_type FROM wp_icl_translations wpml_translations JOIN wp_term_taxonomy tax ON wpml_translations.element_id = tax.term_taxonomy_id AND wpml_translations.element_type = CONCAT('tax_', tax.taxonomy) JOIN wp_icl_translations tridt ON tridt.element_type = wpml_translations.element_type AND tridt.trid = wpml_translations.trid WHERE tridt.element_id IN(189). I've read forum posts about Crowdin, which is where I learned to use CTE and figured out what troubleshooting buttons were safe to press.
<b>Symptoms: </b>
The large sync with Crowdin is completely stuck with missing pages, and most translation jobs seem to be stuck.
<b>Questions: </b>
Why are the translation jobs stuck?
What can I do to resolve the issue of missing pages during the sync? - A few basically are back in that first state where they say they have not been sent for translation.
Are there any specific troubleshooting steps I should follow for large translation jobs?
Is it a problem where I need to optimize the database and what would be the steps if so?
While I consult with our Translation Service experts about your case, please answer the following questions and proceed accordingly so we can better assist you with this issue.
1. Troubleshooting steps
You wrote: "I've tried pressing the no/very-low risk troubleshooting buttons."
Could you please let us know exactly which actions you have already tried?
For example, on the WPML → Support → Troubleshooting page, there are options such as:
- Clear WPML cache
- Remove ghost entries
- Fix post type/taxonomy relationships.
- Assign translation jobs again.
2. Update WPML.
According to the debug information on your site, I can see that our plugins are not up to date with their latest versions. Our core plugin, WPML Multilingual CMS, on your site is currently at version 4.7.6, while we have version 4.8.1 available. If you haven't yet noticed that we have new versions of our plugins, please check the admin screen on your site at hidden link. Click the Check for Updates button, and you will see that new versions are available. You can update our plugins from that screen.
**** Important! Please make a full site backup (files and DB) before you proceed with those steps****
Could you please proceed with updating our plugins and checking if it helps with the issue you encounter on your site?
3. Access to your site and the permission to take a copy.
If the problem persists after updating our plugins to their latest versions and you require further assistance, please share the access details to your site with me. I'm enabling a private message for the following reply. Privacy and Security Policy
We have strict policies regarding privacy and access to your information. Please see: https://wpml.org/purchase/support-policy/privacy-and-security-when-providing-debug-information-for-support/ **IMPORTANT**
- - Please back up the site files and database before providing us access. --
Also, if needed, please allow me to take a copy of your site. For this, I must install a plugin like Duplicator or All-in-One Migration. Please let me know if you agree.
This is Ahmed Mamdouh from the translation proxy team, and I'll help you with this ticket.
I investigated your project, and here are my observations:-
1-I found some stuck/Corrupted jobs; some of them are stuck due to a timeout error from the Crowdin side. I was able to unstuck those jobs, and they are waiting for translation now.
2-I found some other corrupted jobs, some errors happened while sending, and also this happened from the Crowdin side, and I assume it's related to the huge number of jobs, so I canceled them to make sure they are not causing any other issues.
3-I found over 600 Jobs translated and ready to be downloaded into your website, so please go to WPML -> Translation Management and click the "check status and get translations" button, and supposedly you should receive at least 600 completed jobs.
So, I think after my findings/Edits supposedly you wouldn't face any issue syncing completed jobs or sending new jobs.
I recommend not sending a huge number of jobs per batch to avoid timeout errors because the Crowdin API wouldn't be able to handle huge requests.
I have left "Get Translations" to sync for 2 days and none were fetched. I'm sorry I don't know exactly which buttons were pressed, as I mentioned this is my second time around trying to sync with Crowdin, I have only uploaded jobs totaling ~10% of the site's material. I think the debug information I included was before the part of the issue report where it recommended updating to the latest beta, which I have already done. I am fine with granting access.
Please send another ability to send a private message? I'm not getting that error but instead a different error on the production site, even after enabling WP_DEBUG
I checked the production site, and when I clicked the "Check status and get translations" button, I got the same error about the memory as shown in the attached screenshot.
Would you please record a video or provide me with a screenshot to show me what exactly the error you are seeing?
Also, I strongly recommend increasing the memory limit and trying again; this could solve the issue permanently.
I got Flywheel to increase the memory limit for another 18 hours to 512M, and now it syncs and updates but only for a short time period and syncs only a hundred or two ish at a time? But when it hits a stop it still says syncing - as in it will only sync about one or two hundred, and then stop, but then the page looks like it still says fetching translations.
May I ask, do you have a specific job or batch that was marked as completed in Crowdin, and you have difficulty retrieving it?
The loading time is taking too long because you have over 10,000 jobs to check, which is an enormous number for any API to check, and maybe one of the jobs is corrupted and interrupts the loading.
So I recommend tracing the completed jobs, which is going to be much easier to trace and will avoid consuming a lot of time tracing all the sent jobs.
How would I do that? The jobs api call response when fetching translations seems to complete and then show a list of completed jobs without any error and says how many remain.
I think you might've misunderstood, The number of pending jobs when down by about 200 and the call when fetching translations when checking the network consol seems to have completed (as in: that jobs api call returns a bunch of returned translations and finishes up by saying remaining translations: 19025 [approximately]) but I also get an admin ajax error and in the web browser interface it just keeps loading saying that it's still fetching translations even though it isn't
Languages: English (English )Spanish (Español )Italian (Italiano )
Timezone: Europe/Rome (GMT+02:00)
Hello,
Let me explain in a different way. the main issue is that inside the "batch" of translations that you sent, there are many that are let's say "wrong" (that have issues, are corrupt, etc).
We don't know WHICH ones are like that, so we can't just take them out and let the process continue with the other translations.
What my colleague was suggesting was to basically provide him with the names of some samples of the completed translations (a few ones will be ok) that you can find in crowdin (not on the actual website) so we can then investigate further.
It has nothing to do with the API call that you mentioned in the last reply but in the actual platform (Crowdin). You need to get back to their support and ask for samples of the completed jobs and then send them over (we'll also try to contact them in the meantime).
