Ungroupable error stacktraces: can Raven-Ruby leverage Rails.backtrace_cleaner?

[davidstosik]

Switching to Sentry 10 recently, we started having big issues with error grouping in our Rails application. It didn't take us long to notice that the problem seems to come form same errors having different stacktraces.

The difference is located in the name of methods generated from template files.
I couldn't find better documentation or Rails code extracts, so instead allow me to share this RailsConf video extract instead 😬: templates are translated into methods.

The stacktrace would look something like that:

Focus on _app_views_home_index_html_erb___2412767935268124114_70177239986900. This method name is generated from the app/views/home/index.html.erb template. I could not pinpoint in Rails code where those strings of digits (fingerprints?) are generated and appended to the method name, but they seem to vary depending on multiple factors (at least the file's content, but it looks like for a given file we've produced multiple values). This generated in Sentry multiple errors that would not be grouped together.

Here's what a stacktrace diff looks like between two errors I would expect to get grouped together on Sentry, but that did not:

This diff was displayed in Sentry's "Similar issues" tab.

There might be settings on Sentry that would allow those similar issues to eventually look close enough to be merged by Sentry, but I wondered what tools were available on my end to clean that up.

That's when I found about ActiveSupport::BacktraceCleaner. This is what Rails uses to display cleaner, more focused backtraces when an error is displayed.

In fact, Rails::BacktraceCleaner introduces a filter that cleans up the template method names from the backtrace, which is exactly what I was looking for. It also removes a lot of noise which is not really what we're looking for when first debugging the application on Sentry. (Silencers are what splits the "application trace" from the "framework trace".)

I though it would be great if I could just apply Rails.backtrace_cleaner to the stack traces before they get sent to Sentry, but the best thing I found myself able to do (without rewriting that logic), was to write a small monkey patch. 🙈

module Raven
  class Backtrace
    class << self
      alias_method :orig_parse, :parse
      def parse(backtrace, opts = {})
        orig_parse(::Rails.backtrace_cleaner.clean(backtrace), opts)
      end
    end
  end
end

I hijacked the Raven::Backtrace.parse method in order to clean up the backtrace while it's still a raw array of lines, before Raven works on it and produces frames it will send to Sentry. The result, for the same error I screenshot at the top, can be seen in this other screenshot:

That's quite satisfying, and should fix our problems with errors not getting merged together (also should help making the stacktrace displayed on Sentry more workable, by focusing on the parts of the code that are most likely to be producing errors.

Now the part I'm not satisfied about, obviously, is that I had to write a monkey patch. 🐒

My question is: is there any better mechanism (provided by Raven, or else) that would allow an application to apply a BacktraceCleaner to the raw backtrace before it gets parsed by Raven? (I am aware of Raven's processors, but those happen after the stacktrace has been parsed...)

[bblack]

@davidstosik a question unrelated to your issue, but related to your screenshot:

Do you know why Sentry's web app sometimes shows multiple similar errors in a single group? In your screenshot, for example, is shown two different errors, with two different classes: An ActionView::Template::Error and a NameError. I've got a project where this has caused hundreds of different groups to appear in Sentry's web app, where the only difference is the number of such errors displayed in one group, and I cannot figure out whether that's a bug in Sentry's web app, a but in Sentry's client lib, a bug in our application, or everything working as intended and I just can't figure out what that intent is.

Edit: Nevermind, I discovered that this is due to nested errors. If you raise an error B in a rescue block executed due to error A, then B's method #cause will return A, and both will be represented in the same Sentry event. So, intended (undocumented?) behavior, I guess.

[st0012]

@davidstosik thanks for reporting! I think this is an issue that needs to be solved. But using Rails' backtrace_cleaner directly is too risky as they're pretty aggressive and behave quite differently in different Rails versions.

Anyway, I've created #1011 to solve this issue in the least impactful way.