Skip to content
This repository has been archived by the owner on May 2, 2022. It is now read-only.

Sync missing translation keys in app/locales/*.json files to a Google Sheet #481

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open

Sync missing translation keys in app/locales/*.json files to a Google Sheet #481

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
b379cbc
add script for finding missing translations across all locales
michaelmcmillan Mar 29, 2020
06d1ef0
remove unused var
michaelmcmillan Mar 29, 2020
5a50bc1
port lost-in-translation.py to javascript for consistency
michaelmcmillan Mar 29, 2020
004d5e0
ported over script usage
michaelmcmillan Mar 29, 2020
721769d
remove python script
michaelmcmillan Mar 29, 2020
432870d
delimit language with ---
michaelmcmillan Mar 29, 2020
f70d5a6
determine locales from dir
michaelmcmillan Mar 29, 2020
ae00680
upload missing translations to a google sheet
michaelmcmillan Mar 29, 2020
490106c
empty string as missing
michaelmcmillan Mar 29, 2020
6ecc6fa
sync all available sheets
michaelmcmillan Mar 29, 2020
34e600a
add sheet if it doesnt already exist
michaelmcmillan Mar 29, 2020
e2def1e
avoid rate limit
michaelmcmillan Mar 29, 2020
1fe146f
rate limit circumvension
michaelmcmillan Mar 29, 2020
99f1c80
prettify comment
michaelmcmillan Mar 29, 2020
d355fbf
refactored code and made it less complex after feedback from @fossecode
michaelmcmillan Mar 29, 2020
be1a7e1
linted
michaelmcmillan Mar 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Prev Previous commit
Next Next commit
rate limit circumvension
  • Loading branch information
@michaelmcmillan
michaelmcmillan committed Mar 29, 2020
commit 1fe146f88dba2279f1e9aaddfa816983bd7212a6
24 changes: 15 additions & 9 deletions ops/lost-in-translation.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,28 +136,34 @@ for (const locale of allLocales) {
const sheet = doc.sheetsByIndex[sheetIndex];
for (const locale of allLocales) {
if (sheet.title !== locale) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand it correctly, there is one sheet per locale?

Is it then necessary to loop through the sheets here, can't we just loop through the locales and try to add them (from the code it seems like it throws an error if it does not exist)?

try {
          await doc.addSheet({ title: locale, headerValues: ['key', 'translation'] });
        } catch (error) {
          // We don't do anything if the sheet for this locale exists.
        }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct, take a look at the sheet here: https://docs.google.com/spreadsheets/d/1ILFfc1DX4ujMnLnf9UqhwQGM9Ke3s1cAWciy8VqMHZw/edit#gid=462835362

The reason I'm looping over locales is because that way we will create sheets for new locales automatically. There is no way in the API to retrieve the sheet names in the spreadsheet. This way we don't have to worry about keeping locales and sheets in sync (it will sort it self out).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't read your comment thoroughly enough. Yeah, maybe we can get away without that loop.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha getting late... The reason we have to loop over those are because there is no way of retrieving a sheet by its name. Does that make sense? The doc.addSheet doesn't return the sheet if it doesn't exist, it only throws an exception. So we have to, at some point, look at all the sheets to check if they match the current locale we are looping over.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I thought the for-loop ended after adding a sheet. A little hard to wrap my head around the logic, but as long as it works it is more than good enough.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Just refactored it, better now?

try {
await doc.addSheet({ title: locale, headerValues: ['key', 'translation'] });
} catch (error) {
// We don't do anything if the sheet for this locale exists.
}
continue;
}

try {
// Create a sheet if it doesn't already exist.
await doc.addSheet({ title: locale, headerValues: ['key', 'translation'] });
await new Promise(resolve => setTimeout(resolve, 10*1000));
} catch (error) {
// We don't do anything if the sheet for this locale exists.
}

// Add the missing rows by looking at the key column.
const rows = [];
for (const englishTranslationKey of allEnglishTranslationKeys) {
if (!translationKeysByLocale[locale].has(englishTranslationKey)) {
const row = { 'key': englishTranslationKey, translation: '' };
rows.push(row);
}
}

// Print out how many rows we added.
const addedRows = await addUniqueRowsToGoogleSheet(sheet, rows);
console.log(`Added ${addedRows.length} of ${rows.length} missing translations to the ${locale} sheet.`);
}

// Avoid getting rate limited by Google's API by waiting 100 seconds between each sheet.
console.log('Waiting a 100 seconds before processing the next sheet.');
await new Promise(resolve => setTimeout(resolve, 100*1000));
// Avoid getting rate limited by Google's API (max writes per 100 seconds).
console.log('Waiting before processing the next sheet.');
await new Promise(resolve => setTimeout(resolve, 20*1000));
}
}

})();