[pull] main from unicode-org:main #16
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… dayPeriods; add tests & spec note (#3857)
…r-apps/js (#3771) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…gin from 3.1.0 to 3.3.0 in /tools (#3839) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…collision for it (#3867)
and update LDML radical-stroke documentation
…standard out noise (#3965) I started this ticket because I was seeing a lot of noisy warnings and errors in the regular tests -- I ended up in a rabbit hole with the generated population data. This change updates the data inputs and fixes errors in the scripts so we can regenerate population data in a stable way now. ### Scripts ran: * mvn package -DskipTests=true * Re-ran these scripts, they need to be run more regularly, some changes happen * java -jar tools/cldr-code/target/cldr-code.jar AddPopulationData # Runs successfully now, some changes happen * java -jar tools/cldr-code/target/cldr-code.jar ConvertLanguageData # Runs successfully now, some changes happen * These scripts do not run * java -jar tools/cldr-code/target/cldr-code.jar WikipediaOfficialLanguages * java -jar tools/cldr-code/target/cldr-code.jar GenerateMaximalLocales * Running tests on github (I still cannot locally run all of the tests* #### Script output changes A lot of the script Standard Out messages mentioned in the original ticket are now fixed and will not appear -- mostly from fixing input data sources and a few processing scripts. If there are legitimate errors in the future the warnings and errors will appropriately come back. * Suriname had 2 un-distinguished sources of literacy data, this will now take the max value of the two * one was the overall number * the other had filtered institutional data * Since the aggregate regions from `world_bank_data.csv` are now gone, there are no more warnings about aggregates without country codes, eg. "Sub-Saharan Africa (all income levels)` ### Data changed: * `country_language_population.tsv` * Fixed some areas where spaces were used that should the tabs -- this affected how scripts parsed Kara-Kalpak, bug introduced in #3657 * Added `Cantonese (Traditional) yue` row otherwise `yue` would disappear in the re-generated `supplementalData.xml` -- introduced in #3945 * `factbook_gdp_ppp.csv` & `factbook_gdp_ppp.csv`: CIA Factbook data updated and imported using the csv that's exported by the [CIA's website](https://www.cia.gov/the-world-factbook/references/guide-to-country-comparisons/) -- see also [the old CLDR update documentation](https://cldr.unicode.org/development/updating-codes/updating-population-gdp-literacy). * This will update all population counts in `supplementalData.xml` * Some stale data was removed from the Factbook there I added missing entries to `other_country_data.txt` * `other_country_data.txt`: Added information that used to be in earlier versions of the CIA Factbook * `world_bank_data.csv`: Re-generated from [the World Bank Website](https://databank.worldbank.org/reports.aspx?source=world-development-indicators#) . See also [the old CLDR update documentation](https://cldr.unicode.org/development/updating-codes/updating-population-gdp-literacy). * A big difference is that I correctly read the instructions and did not import the country aggregates, eg. "Sub-Saharan Africa (all income levels)` * `alternate_country_names.txt`: Removed no longer needed skipped names since we no longer import CIA Factbook aggregates ### Consequences for`supplementalData.xml` * **Official Languages**:`<language>` territories tag should be the territories where the language is **official** -- so some entries updated. For instance Mocheno was incorrectly considered an official language of Italy in #3665 * **Population counts** are incremented, so some **language population percentages** may increase or decrease if the input data is absolute value (since the denominator changed) * **GDPs** also changed * **Literacy Rates** some have changes * Note there was a wonderful bug where the UN literacy data was mis-parsed, so "96%" would be mis-read as "0.96%" -- I fixed that * **References**: The two Kara-Kalpak references are now grouped correctly, Chinese reference has been given more context too
- see CLDR-17906 for further work on parent locales
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )