BUG Fix utf8 non-latin characters generating invalid encoding #9764
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to silverstripe/silverstripe-graphql#318
I have tracked a text encoding issue where DBHTMLText::Plain() was mangling encoding of utf-8 non-latin text.
I have added a unit test, but I couldn't quite replicate the bug in the test without dumping the entire content area of my client's website (which, for confidentiality cases, obviously I can't push to github).
This fix replicates the same UTF-8 treatment in Convert::nl2os, which also adds this flag.
Without this fix, elemental module crashes the CMS when dealing with asian characters. I would put this at a high severity. :)