Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: panic, when using utf-8 #47

Closed
IAD opened this issue Jan 18, 2023 · 2 comments
Closed

bug: panic, when using utf-8 #47

IAD opened this issue Jan 18, 2023 · 2 comments
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@IAD
Copy link

IAD commented Jan 18, 2023

Describe the bug

use the test:

func TestDefaultDriver_UTF8(t *testing.T) {
	detector := goaway.NewProfanityDetector().WithCustomDictionary(
		[]string{"anal", "あほ"}, // profanities
		[]string{"あほほ"},        // falsePositives
		[]string{"あほほし"},       // falseNegatives
	)

	message := "いい加減にしろ あほほし あほほ あほ anal ほ"
	expected := "いい加減にしろ **** あほほ ** **** ほ"

	isProfane := detector.IsProfane(message)
	require.True(t, isProfane)

	censored := detector.Censor(message)
	require.Equal(t, expected, censored)
}

What do you see?

panic at:
github.com/TwiN/go-away.(*ProfanityDetector).Censor(0xc000082f08, {0xf8f7c7, 0x3c})
.../github.com/TwiN/go-away/goaway.go:193 +0x7a5

What do you expect to see?

UTF-8 filtered as well as ASCII

List the steps that must be taken to reproduce this issue

use the test from the description

Version

v1.6.9

Additional information

No response

@IAD IAD added the bug Something isn't working label Jan 18, 2023
@TwiN
Copy link
Owner

TwiN commented Jan 18, 2023

Looks like a duplicate of #43 -- which likely has to do with the fact that UTF-8 characters can be more than one byte.

@TwiN TwiN added good first issue Good for newcomers help wanted Extra attention is needed labels Feb 18, 2023
@stephenduke-care
Copy link
Contributor

stephenduke-care commented Sep 8, 2023

I attempted a fix on this here: #64

@TwiN TwiN closed this as completed Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants