[BUG] The output byte buffer is too small to contain the encoded data, encoding codepage '65001'

[BakaFT]

What happened?

Using Obsidian V5.1.6, trying to open rcp-fe-lol-champ-select\assets.wad (of Tencent Client Build), and get this error:

2023-07-06 15:01:39.586 [INF] Opening WAD files: ["D:\\WeGameApps\\英雄联盟\\LeagueClient\\Plugins\\rcp-fe-lol-champ-select\\assets.wad"]
2023-07-06 15:01:39.592 [ERR] Error
System.ArgumentException: The output byte buffer is too small to contain the encoded data, encoding codepage '65001' and fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
   at System.Text.Encoding.ThrowBytesOverflow()
   at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar)
   at System.Text.UTF8Encoding.GetBytes(ReadOnlySpan`1 chars, Span`1 bytes)
   at XXHash3NET.XXHash64.Compute(ReadOnlySpan`1 data, UInt64 seed)
   at LeagueToolkit.Core.Wad.WadFile..ctor(FileStream stream)
   at Obsidian.Shared.WadExplorer.<>c__DisplayClass46_0.<OpenWad>b__0() in D:\a\Obsidian\Obsidian\Obsidian\Shared\WadExplorer.razor.cs:line 96
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Obsidian.Shared.WadExplorer.OpenWad() in D:\a\Obsidian\Obsidian\Obsidian\Shared\WadExplorer.razor.cs:line 92

Expected behavior

Open successfully

Reproduction steps

:) as above

Version (bug)

5.1.6

Version (working)

No response

[BakaFT]

https://github.com/Crauzer/XXHash3.NET/blob/main/XXHash3.NET/XXHash64.cs#L21
🤔Too many chinese characters in this file? If I understand right, this code assume all chars are 2-byte, but many chinese characters are 3 or even 4 bytes in UTF-8 encoding.

Since the max size of a UTF-8 character is 4 bytes, will this fix it? (of course it's a waste of ram)

Span<byte> encodedData = stackalloc byte[data.Length*4];

[Crauzer]

Thanks for reporting the issue. I am aware that the code for allocating the UTF-8 buffer in XXHash3.NET is not correct.

The conversion should try to allocate the maximum possible amount of bytes for the input string.

[Crauzer]

Fixed in #211