Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue with interpolation of strings vs charlists. Fixes #729. #733

Merged
merged 1 commit into from
Jan 18, 2020

Conversation

TheFirstAvenger
Copy link
Contributor

@TheFirstAvenger TheFirstAvenger commented Jan 16, 2020

This PR fixes #729. This was a fun one to track down. When we pass code into :elixir_tokenizer to identify the token locations, we first convert the string to a charlist:

credo/lib/credo/code.ex

Lines 132 to 134 in d77eab6

source
|> String.to_charlist()
|> :elixir_tokenizer.tokenize(1, file: filename)

This results in column numbers consistent with the charlist indexes of the characters. For the unicode character in question (🇿🇼), it translates to two values:

iex(1)> String.to_charlist("🇿🇼")
[127487, 127484]

This means that when the :elixir_tokenizer identifies the # token in this string:

    source = ~S"""
    "🇿🇼 #{String.upcase(env)}"
    """

it identifies it at the 5th column in the output, which is correct because it is the 5th item in the charlist:

   {:bin_string, {1, 1, nil},
    [
      "🇿🇼 ",
      {{1, 5, nil}, {1, 25, nil},

The problem is that our code to split and replace that interpolation uses String.slice, which uses the String indexes, not charlist indexes, so when it grabs the 5th character, it is off by one and starts the replacement at the { instead of the #. The resulting code has an unexpected/unparsable #:

     Assertion with == failed
     code:  assert expected == InterpolationHelper.replace_interpolations(source, " ")
     left:  "\"🇿🇼                      \"\n"
     right: "\"🇿🇼 #                     \n"

The fix was simply to convert the strings to charlists and use Enum.slice instead of String.slice to replace the values, as the numbers we are working with are the charlist indexes, not the string indexes.

@rrrene
Copy link
Owner

rrrene commented Jan 18, 2020

@TheFirstAvenger Nice detective work! 👍

@rrrene rrrene merged commit 58c1df1 into rrrene:master Jan 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unexpected parser error
2 participants